POC AI-Powered System for Detecting License Plate Violations on Vehicles.
The following proposal is submitted on behalf of Think and Dev, a pioneering technology company focused on blockchain innovation. We are seeking funding to develop an advanced image recognition system with artificial intelligence (AI) capabilities, integrated into Cartesi’s infrastructure.
Core Concept and purpose statement
Project Description
The proposed project aims to build an on-chain-verifiable image recognition system using AI algorithms, which can be run on the Cartesi Virtual Machine and whose results can be stored in the blockchain. The first use case of the system will be to identify number plates of vehicles in violation of the law. When a number plate is identified, it will be stored on-chain ensuring irrefutable proof of the offense.
The ultimate goal of this project is to provide full documentation, open-source code, and pre-built Wheels to encourage knowledge sharing and create an open-source repository. We are committed to delivering quality and well-documented code that leverages Cartesi’s capabilities to address real-life problems using cutting-edge technology. In addition, we will include pre-built Wheels to facilitate seamless installation and usage of our software by the community.
Also, as we advance with the project, we will be trying to generate examples, extract snippets of code and generate discussions on discord that could be useful for the community to understand different Cartesi use cases.
We will prove that we can improve traffic safety by using blockchain and IA to automate the detection of number plates and the identification of traffic violations.
Project Goals
-
Milestone 1: Create a Proof of Concept (PoC) demonstrating the capabilities of our AI-based traffic violation recognition system when executed within the Cartesi Machine. This PoC will showcase the power of Cartesi by running a trained model (using the YOLO object detection algorithm and character recognition systems) that can accurately identify license plates in images of vehicles involved in traffic offenses. The focus will be on the practical application of Cartesi for real-time predictions, excluding the resource-intensive model training phase.
-
Milestone 2: Build a user-friendly interface that seamlessly integrates image input into the Cartesi Machine. This interface is vital for ensuring a smooth and secure process of processing and storing images, along with their associated license plate numbers, on the blockchain. We will ensure that anyone can easily retrieve this information by making a simple GraphQL query to the notice list generated by our Decentralized Application (DApp). This milestone will serve as a practical example of Cartesi’s utility in enhancing data accessibility and management.
-
Milestone 3: Foster community learning and engagement by providing comprehensive documentation and open-source code for the image recognition system. Our commitment to transparent documentation will empower developers and enthusiasts within the Cartesi community to comprehend and utilize the technology for diverse applications. By sharing our findings and experiences openly, we aim to catalyze discussions and knowledge-sharing within the Cartesi ecosystem, showcasing its potential for a wide range of projects beyond our traffic violation recognition system.
How you will use Cartesi, specifically?
By leveraging Cartesi’s innovative Virtual Machine, the process of identifying the number plate of violating vehicles through AI models reaches new heights of efficiency and security. By executing the AI model within the Cartesi Virtual Machine, we ensure swift and accurate processing of data, enabling seamless identification of number plates.
One of the most significant advantages of employing Cartesi is its integration with the blockchain, where the results generated by the AI model are securely stored. This integration creates a tamper-resistant and unalterable record, establishing a level of transparency and trustworthiness that traditional systems cannot match. Embracing Cartesi technology empowers us to maintain a distinct and immutable trail of information, paving the way for enhanced accountability and reliability in the identification and management of violating vehicles.
The following diagram shows the workflow from the upload of the image to the application until the infraction is reported by the user.
-
The user uploads a photo of the vehicle in infraction to the App.
-
In the App the photo is transformed into bytes.
-
The encoded image is sent as input to the DApp via a payload.
-
The input data is analyzed through the execution of Python scripts that retrieve the image and process it to obtain the bounding box containing the license plate (object detection algorithm, YOLO) and the characters contained in it (character recognition algorithm, OCR).
-
The Notice generated by the DApp is obtained from the Cartesi node.
-
The byte-encoded image (or it’s hash) and the license plate number obtained are returned to the app.
-
The infraction notification is sent to the authority and, if possible, to the infractor.
-
The infractor can verify the computation and present a fraud proof if he/she found flaws. If the infractor doesn’t complain within a given challenging period (Cartesi’s dApp epoch), the computation is confirmed on-chain. In case the result is disputed, the Cartesi’s interactive dispute resolution system hops in and ensures the correct outcome.
-
Once the final results are obtained, the Notice is confirmed on-chain.
At a future stage, a voucher could be created that adds the user’s wallet to a smart contract, ensuring that, when the offender pays, payment is received directly by the complainant with an on-chain protocol. Using cartesi vouchers, it could be possible to interact with base chain to ensure that the payment flow of the fine is also recorded on-chain.
Technical details
The license plate recognition model will be built by harnessing the synergistic potential of OpenCV, YOLO, and Optical Character Recognition (OCR). OpenCV, a versatile open-source library for computer vision and image processing, will empower the model with efficient image manipulation, key feature extraction, and vital preprocessing capabilities. YOLO (You Only Look Once), a cutting-edge object detection framework, renowned for real-time precision, will swiftly identify license plates in images. The integration of OCR will further enhance the model’s ability to accurately recognize and decipher characters on license plates, ensuring comprehensive and swift results.
Value Proposition
The successful execution of the POC can pique the interest of the community and other developers for the following reasons:
-
Accompanying AI progress: In light of the swift and constantly evolving progress of artificial intelligence models, it becomes increasingly crucial to demonstrate Cartesi’s ability to harness and leverage these cutting-edge technologies. By showcasing Cartesi’s potential to integrate with new and revolutionary AI advancements, we not only inspire the community but also underscore the platform’s capability to drive innovation and stay at the forefront of transformative developments.
-
Showcasing Real-World Use Cases: By addressing a practical problem in traffic management and law enforcement, the project showcases the versatility and relevance of Cartesi’s technology beyond theoretical concepts.
-
Community Engagement and Adoption: The application’s citizen reporting feature fosters community engagement and encourages public participation in addressing traffic violations. Developers interested in building applications with community-driven components may find inspiration from this aspect of the project.
Estimated Duration and Funds Requested
Duration: 1 month
Total Funds request: $5000
-
Milestone 1: 1500
-
Milestone 2: 1500
-
Milestone 3: 2000
Subsequent Vision and Extensibility
If the proof of concept is successful, it will be possible to move forward with developing and implementing a large-scale project to detect vehicle violations. This project aims to transform traffic management by offering an effective, transparent, and secure solution for both citizens and authorities. The following steps will be taken to achieve this goal.
Video-Based Detection
In this step, the application extends its capabilities to process recorded video footage for infraction detection. The algorithm developed in the previous step is adapted to analyze video frames systematically and identify license plate violations within the recorded content.
Infraction Detection
Building upon the achievements of the POC, the next crucial stage focuses on enhancing the application’s capabilities to detect violations from static images. Advanced computer vision techniques and efficient processing algorithms are implemented to analyze images and identify vehicles with infractions accurately.
Integration of Infraction Database
In the third step, the application incorporates an infraction database containing a list of license plates associated with arrest warrants or other legal actions. Whenever a license plate match is found between the detected vehicles and the entries in the infraction database, appropriate alerts or notifications are triggered.
User-Friendly Interface and Rewards
Moreover, we propose that users who provide convincing evidence leading to the identification of offending vehicles can be rewarded with a portion of the fine imposed for the offense. This innovative approach not only encourages community participation but also establishes a collaborative and community-driven framework for traffic enforcement.
Reusability and Other Use Cases
The reusability of this vehicle license plate recognition program offers a wide range of possibilities for applications in different contexts and sectors. Thanks to its modular architecture and the use of open-source technologies, the program can be adapted and reused to address various issues in today’s society. Some prominent use cases include:
-
Traffic Management: In addition to detecting traffic violations, the system can be used to monitor and improve traffic flow in urban areas and highways. Identifying illegally parked vehicles, analyzing traffic patterns, and measuring vehicle flow are some examples of how this solution can contribute to more efficient and safer traffic management.
-
Public Safety: Law enforcement agencies can utilize the program to identify vehicles involved in criminal activities or with pending arrest warrants. By integrating the infringement database with police records, surveillance and response to criminal incidents can be enhanced, increasing safety in communities.
-
Parking Lots and Tolls: The system can be applied in public and private parking lots to facilitate vehicle entry and exit through automatic license plate recognition. Additionally, in road toll systems, it can be used to expedite fee collection without the need for vehicle stops.
-
Logistics and Fleet Management: In the logistics and transportation sector, the program can be used to track and manage vehicle fleets, ensuring more effective control over the delivery of goods and itinerary tracking.
-
Residential Parking Security: In residential areas, the system can be employed to detect unauthorized vehicles parked in private spaces, providing better management and security for residents.
-
Environmental Surveillance: By integrating surveillance cameras with the system, it becomes possible to identify polluting vehicles or those in violation of environmental regulations, facilitating the implementation of measures to protect the environment.
-
Industrial Automation: In the industry, the program can be used to identify and track vehicles within industrial complexes, enhancing logistical management and security in these areas.
Risks and Contingency Plans
If the license plate detection algorithm’s performance and accuracy fall below expectations, we may need to explore alternative AI techniques or models. In some cases, we may need to implement an algorithm to improve the quality of the images for more accurate recognition of the number plate.
Success criteria.
In order for the proof of concept to be considered a success, the following conditions must be fulfilled:
-
The model running on the Cartesi Virtual Machine must achieve a high level of accuracy in identifying license plates from images, exceeding a certain pre-set threshold.
-
The results obtained must be successfully stored on-chain.
Final report
The Think and Dev team is committed to give this code to the community. In case we fail or accomplish the goals, we leave the project open to whoever wants to take it up, fork it, complement it, adapt it, and evolve it.
About Your Development Team
- Lucas Marc
Blockchain Technical Leader, working on Think and Dev leading multidisciplinary teams in different verticals.
- Juan Manuel Aragón Paz Ph.D
Ph.D. in Geophysics. Knowledge of GNSS processing, Python, and data science and analysis. Graduate of the National University of La Plata. More than 10 years as a teaching assistant in different subjects. His first WP was published last April.
- María Mercedes Vazzano Ph.D
PhD in Astronomy. Experience in the use of Python libraries for data management and visualization tools, as well as in the implementation of computer vision models. Graduate of the National University of La Plata.
ERC-20 Payee address
0x5d7224E3A04384c44E2De3ed5D135181d0f3a87a