This thesis explores two distinct approaches to optimizing machine learning for resource-constrained environments: hierarchical machine learning (HiML) for energy-efficient classification and neural network-based spectral reconstruction for cost-effective computational spectrometers. Both topics address different challenges but share the common goal of improving computational efficiency for embedded systems. The first part investigates hierarchical machine learning as a means to reduce energy consumption in classification tasks. Unlike traditional flat classifiers, which apply equal computational effort to all inputs, hierarchical machine learning structures classification as a hierarchy of decisions, allowing for early exits when classification confidence is high. Experimental results demonstrate that this approach reduces energy consumption by up to 47.63% while maintaining competitive accuracy. Additionally, reinforcement learning is introduced as an optimization strategy for classifier selection, significantly accelerating the search process compared to exhaustive methods. The second part focuses on spectral reconstruction, where neural networks recover full spectral information from complex sensor outputs. A physics-informed data augmentation method is introduced, leveraging the optical properties of transmission spectra to expand the dataset from 214 to over 10,000 samples. This augmentation improves model generalization without requiring additional physical measurements. Furthermore, layer-wise relevance propagation is applied to optimize the sensor hardware, enabling a 94% reduction in input dimensions while preserving high reconstruction accuracy. Benchmarking results reveal that smaller, well-optimized models can match or even surpass the performance of larger networks, demonstrating the potential for efficient, compact spectral reconstruction systems. The findings of this thesis contribute to the development of energy-efficient and computationally lightweight machine learning solutions. The proposed approaches enable more effective classification pipelines and high-fidelity spectral reconstruction on embedded systems, paving the way for real-world applications in industrial monitoring, portable spectral sensing, and other edge AI scenarios.