Ongoing  /  Dar SEED Innovation  /  Bangalore

Real Time
Sensing

Urban designers are trained to read the city through what we can see: form, density, movement, light. What we rarely capture is what the city sounds, feels, and breathes like in real time. That is the gap this project is trying to close.

Platform
NVIDIA Jetson Orin Nano Super
Edge inference, on-device
Sensors
GPS · Temperature · Humidity
MEMS microphone · PM2.5 / PM10
IMX477 camera
Recognition
SEED Chairman Award
Sounds of the City
Status
Five build updates published
Site selection under way

Urban designers shape cities without ever really knowing what a street feels like. They work from maps, simulations, and averaged datasets. None of that tells you whether a specific street is too loud, too hot, or why people avoid it.

This device is built to fill that gap. Five sensors, sound, temperature, humidity, air quality, and video, all tied to the same GPS coordinate and the same timestamp. Not five separate data streams. One synchronized observation of a place at a moment in time.

The point is coverage density. Existing sensor infrastructure is real-time but sparse. The data exists but it does not exist where decisions actually need to be made.

Sound and video are not independent data points here. They contextualize everything else. High particulate matter means something different on a street choked with traffic than on one that looks empty. The camera and microphone explain the reading.

The use case is diagnosis before design. Before an urban designer intervenes on a street, they need to know what is actually wrong with it and why. Right now that answer comes from simulation and intuition. This makes it measurable.

Coming from a background in architecture and urban design, with a specialisation in machine thinking urbanism, I have spent years thinking about how cities work, analysing spatial data, building models. But I had never built physical hardware before. Building the instrument that creates the dataset is a completely different discipline, and a humbling one.

The system

Six inputs into one edge device, three classes of output. Everything writes timestamped to an SD card so the readings stay tied to each other.

Selected for SEED

SEED Chairman Award, Sounds of the City

Sound is a powerful yet overlooked layer of urban data. From the quiet hum of residential streets to the intensity of commercial corridors, Sounds of the City explores how we can capture, analyse, and interpret these soundscapes to better understand how cities function, feel, and evolve.

The project was selected for the incubation stage of Dar's SEED Innovation programme, and later received the SEED Chairman Award. The aim is to build high-resolution sound datasets that can inform urban design, planning decisions, and policy, opening up new ways of reading the city beyond what we can see.

With thanks to Mustapha Madi · Dan Horner · Tom Benson · Payel Mallick · Treveena Rofaeel · Noorhan Mustafa Hanafi · the SEED jury

Build log

Documenting every step. The wins and the walls. Because the goal is not just a dataset at the end. It is showing that people who understand cities can also build the tools to listen to them.

Update 01
A breadboard and wires going in every direction

That is a breadboard. Those are jumper wires going in every direction. And yes, I had to look up what a jumper wire was.

Three sensors working on the Jetson: the GPS module locking onto satellites and reading coordinates, temperature and humidity giving live readings, and a MEMS microphone capturing audio.

GPS module locking
Temperature + humidity live
MEMS microphone capturing
First working configuration. GPS, temperature, humidity and audio on one board.
Update 02
Five senses, one coordinate, one timestamp

Sound, temperature, humidity, air quality, and video, all tied to the same GPS coordinate and the same timestamp. Not five separate data streams. One synchronized observation of a place at a moment in time.

Existing sensor infrastructure is real-time but sparse. The data exists but it does not exist where decisions actually need to be made.

Update 03
The device left the desk

A trial run right outside the office. A few weeks ago this was a breadboard with wires going in every direction. Now it is out on a street in Bangalore, reading it.

Every detection is written to a CSV with a timestamp, object class, confidence score, bounding box coordinates, and GPS position. Temperature, humidity, and timestamp are overlaid directly on the video feed, all saved to the SD card.

Camera IMX477 at 1920x1080
Pipeline GStreamer into YOLOv8
Inference every 3 frames at 15fps
Confidence 0.4
Open problem GPS fix was not locking consistently outdoors during this run. That is the next problem to solve before a proper deployment.
First street run. Detections, temperature, humidity and timestamp burned into the feed.
Update 04
The camera and microphone would not coexist

The reason turned out to be a boot configuration problem rather than anything in the hardware itself. Both sensors need to be registered at startup. When the system configured the microphone, it was overwriting the camera's registration in the process. So whichever sensor got configured last would work, and the other would silently disappear with no error to explain why.

The fix was making sure both sensors were registered together in the same startup configuration, then making it persistent so the setup survives every reboot. Both are now running together correctly.

Open problem The INMP441 picks up everything indiscriminately: fan noise, keyboard clicks, ambient hum indoors. Outside in an urban environment it will be considerably worse. Useful audio classification needs serious filtering work first.
Update 05
Air quality, and a wiring conflict

The air quality sensor is connected and reading PM2.5 and PM10. Getting there was less straightforward than I expected.

The PM2.5 sensor communicates over UART, a serial protocol the Jetson uses to talk to peripherals. The GPS module also uses UART, and the Jetson has a limited number of these serial lines. They could not both be connected the way I had planned. The power situation made it worse: the GPIO pins were already fully occupied, with no spare power pins to simply add another module.

The fix was to route the PM2.5 sensor through USB instead. A small converter module sits in between, translating the sensor's serial output into something the Jetson can read over a standard USB port. It adds a component to the setup but it works, and it keeps the GPS on its own dedicated line.

PM2.5 is fine particulate matter, small enough to enter the lungs. PM10 catches the coarser particles. A reading tied to a GPS coordinate, a timestamp, a temperature, a noise level, and a video frame is a different kind of data point than a reading on its own.

Six inputs on one board. The PM2.5 sensor routed over USB to free the GPS serial line.

Where it is now

Audio

Fine-tuning the classifier on the sounds that actually matter, rather than everything the microphone happens to hear.

Site selection

Choosing deployment sites using readings from sensors already installed across Bangalore, so the instrument goes where the existing network is thinnest.

Still open

Consistent outdoor GPS lock, and filtering the microphone enough to make street-level audio classification usable.

Follow the build updates ↗ Get in touch