This work presents a compact CMOS Image Sensor (CIS) architecture enabling embedded object recognition facilitated by a dedicated end-of-column Compressive Sensing (CS),- reducing on-chip memory needs. Our sensing scheme is based on a combination of random modulations and permutations leading to an implementation with very limited hardware impacts. It is designed to meet both theoretical (i.e.,- stable embedding,- measurements incoherence) and practical requirements (i.e.,- silicon footprint,- power consumption). The only additional hardware compared to a standard CIS architecture using first order incremental Sigma-Delta (sigma delta) Analog to Digital Converter (ADC) are a pseudo-random data mixing circuit,- an in-sigma delta ± 1 modulator and a small Digital Signal Processor (DSP). On the algorithmic side,- three variants are presented to perform the inference on compressed measurements with a tunable complexity (i.e.,- one-vs.-all SVM,- hierarchical SVM and small ANN with 1-D max-pooling). An object recognition accuracy of ? 98.8 % is reached on the COIL database (COIL,- 100 classes) using our dedicated Neural Network classifier. We stress that the signal-independent dimensionality reduction performed by our dedicated CS scheme (1/480 in 480 × 640 VGA resolution case) allows to dramatically reduce memory requirements mainly related to the remotely learned coefficients used for the inference stage. © 2004-2012 IEEE.