🥼 Smart Surveillance System: AI-based Uniform and Face Identification for Lab RES Tool Room Security

🌐 Project Domain

Our “Labcam” system stands at the forefront of cutting-edge IoT-based smart surveillance and security. We’ve engineered a robust solution specifically designed to fortify the security of the Robotics and Embedded System (RES) Laboratory’s tool room, leveraging the power of Artificial Intelligence for real-time uniform and facial identification.

🧑‍🤝‍🧑 Meet Our Team

🖥️ Website Developer: AHMAD IZZAT ALBI ISTAFA (225150207111013)
🤖 AI Engineer: NUR ADIYANTO KUSUMA NUGRAHA (225150301111012)
🔧 Hardware Engineer: ZAHRA AULIAUL ROHMAN (225150301111015)
🤖 AI Engineer: ILMAM HASHFI FIRJATULLAH (225150301111016)
🔧 Hardware Engineer / 3D Designer: YUDANERU VEBRIANTO (225150301111019)

❗ Problem Statements

The increasing number of theft cases, including in academic environments, highlights serious security concerns.
Traditional security systems often rely on manual methods like physical keys and limited human supervision, proving to be ineffective in detecting and preventing threats.
There is a need for a more sophisticated and proactive security solution to prevent unauthorized access and theft in laboratory tool rooms.

🎯 Goals

Analyze the effectiveness of face recognition and uniform detection technology in identifying individuals with authorized access to the laboratory.
Develop and test the integration of face recognition and uniform detection technology to reduce potential unauthorized access and improve real-time security response.
Evaluate the performance of the Smart Surveillance System in accurately distinguishing between authorized and unauthorized individuals with optimal accuracy.

💡 Solution Statements

Develop a Smart Surveillance System based on AI capable of real-time face and uniform detection to enhance laboratory tool room security.
Implement face recognition technology to identify individuals entering the laboratory and uniform detection to ensure only authorized personnel can access the tool room.
Integrate the system with Website for real-time surveillance, Blynk for real-time notifications and a buzzer for immediate alerts when unauthorized access is detected.
Utilize a hybrid AI-IoT approach, with ESP32-CAM for local uniform detection and AWS EC2 for cloud-based face recognition, to optimize performance and bandwidth.
Design the system to categorize outputs into “assistant lab” and “unknown person”.

🧰 Prerequisites – Component Preparation

🔌Hardware Powerhouse:

ESP32-CAM: Camera module and main processing unit.
5V Power Supply: Supplies power to ESP32-CAM and circuit.
Buzzer: Actuator for warnings/alarms (when the subject is not recognized).
Red LED: Illuminates when WIFI is not connected.
Green LED: Illuminates when the subject is recognized.

🧠 Software & Platforms:

Arduino IDE: For ESP32 programming in C++.
TensorFlow: For deep learning models.
AWS (EC2, IoTCore, Lambda, Api Gateway): For cloud server & data communication.
Python: For face recognition AI.
C++: Core logic implementation on ESP32-CAM and for uniform detection AI.
Blynk: IoT platform for notifications and monitoring.
MobileNetV2 0.35: Lightweight model for uniform detection.
HOG & CNN ResNet34 from DLIB: For face detection and feature extraction.
React Website: Website for easy real-time monitoring

🧩 Schematic

🎬 Demo and Evaluation

Demo: Demonstrate real-time detection of uniforms and faces, showing the system’s response (LEDs, buzzer, and Blynk notifications) for both authorized and unauthorized individuals.
🖥️ Watch the demo here:
https://drive.google.com/file/d/1e-iHFh_RC8aGAlzBdJ5NXdDrMfDgr70g/view?usp=drive_link
🔬 Evaluation: The system was tested using Unit Testing and Integration Testing to ensure proper module and system integration.
- Uniform Detection Accuracy: Achieved 88.8% F1 score.
- Face Recognition Accuracy: Achieved 97% accuracy.
- Uniform Detection Response Time (Local): Average of 757 ms.
- Face Verification Response Time (Cloud): Less than 2000 ms.
- Blynk Notification Response Time: Less than 2 seconds.

✅ Conclusion

This project successfully developed a Smart Surveillance System based on AI that effectively identifies uniforms and faces in real-time to enhance laboratory tool room security. The system accurately differentiates between known, unknown, and no-person categories, demonstrating an 89% accuracy for uniform detection and 97% for face recognition. Its hybrid architecture, combining local and cloud processing, ensures high accuracy and low latency, proving effective in preventing unauthorized access through quick responses via Blynk notifications and a buzzer. The system’s advantages include real-time detection, rapid response, optimized bandwidth through a hybrid AI-IoT approach, high accuracy with deep learning models, and adaptability via Agile Development.

🏆 Achievement

🥉 3rd Place Winner at Capstone Graduation Showcase 2025 – AIoT Theme
The Smart Surveillance System project was awarded 3rd place at the Capstone Graduation Showcase 2025, held by FILKOM UB, under the AIoT (Artificial Intelligence of Things) category. This achievement highlights the project’s success in integrating AI and IoT technologies into a practical, efficient, and impactful security solution for laboratory environments.

💰 Budget Efficiency Milestone
The entire system was successfully developed with a budget of approximately IDR 150,000, demonstrating the team’s strong capability in cost-effective innovation. By utilizing open-source tools, affordable hardware (like the ESP32-CAM), and cloud-based AI processing, the project proved that high-performance smart surveillance can be built on a minimal budget—making it a viable option for educational institutions and low-resource environments.