The models are becoming larger, and the data sets are larger, leading to greater success in deep learning, but this approach is not compatible with many contexts that are resource-constrained. There are two scarcity types that are prevalent: Limited compute, memory, and energy on edge, mobile, and embedded devices, and limited labelled data in specialised domains where it is costly to annotate it. The two main strategy families that enable deep learning under these constraints are discussed: model compression (compressing the trained network to make it more economical), and transfer learning (reusing knowledge from data-rich source tasks to require relatively little target data). It presents the field through four lenses: a taxonomy of compression techniques: pruning, quantization, knowledge distillation, and compact architectural design, as well as the transfer-learning techniques covered: few-shot learning, domain adaptation, parameter-efficient adaptation, fine-tuning, and feature extraction, which tackle data scarcity issues, a deployment pipeline that fuses the two: (few-shot learning, etc. to overcome limited data and compression, etc. to overcome limited compute) prior to on-device deployment, and open challenges, such as accuracy-efficiency trade-offs, automation, robustness, and evaluation. It shows that compression and transfer are complementary, both for orthogonal axes of scarcity: the combination of a large, general-purpose, pre-trained model and its compression for the target device and task is the most prevalent for practical low-resource deployment. The design principle is always to align the strategy to the binding constraint either transfer learning when data is scarce, or compression when compute is scarce, or both when both data and compute become scarce just like they do more often than not.