POC AI-Powered System for Detecting License Plate Violations on Vehicles.
The following proposal is submitted on behalf of Think and Dev, a pioneering technology company focused on blockchain innovation. We are seeking funding to develop an advanced image recognition system with artificial intelligence (AI) capabilities, integrated into Cartesi’s infrastructure.
Core Concept and purpose statement
Project Description
The proposed project aims to build an on-chain-verifiable image recognition system using AI algorithms, which can be run on the Cartesi Virtual Machine and whose results can be stored in the blockchain. The first use case of the system will be to identify number plates of vehicles in violation of the law. When a number plate is identified, it will be stored on-chain ensuring irrefutable proof of the offense.
The ultimate goal of this project is to provide full documentation, open-source code, and pre-built Wheels to encourage knowledge sharing and create an open-source repository. We are committed to delivering quality and well-documented code that leverages Cartesi’s capabilities to address real-life problems using cutting-edge technology. In addition, we will include pre-built Wheels to facilitate seamless installation and usage of our software by the community.
Also, as we advance with the project, we will be trying to generate examples, extract snippets of code and generate discussions on discord that could be useful for the community to understand different Cartesi use cases.
We will prove that we can improve traffic safety by using blockchain and IA to automate the detection of number plates and the identification of traffic violations.
Project Goals
-
Milestone 1: Build a PoC of the image recognition system that can be executable within the Cartesi Machine. A model that can identify a license plate in an image of a vehicle involved in an offense will be trained using the YOLO object detection algorithm and character recognition systems. The Cartesi Machine will be employed for predictions, excluding model training due to the substantial computational demands associated with this process.
-
Milestone 2: Develop a highly efficient system that allows for easy image input into the Cartesi Machine. This is crucial to ensure that all processing is executed flawlessly and that the image, along with its license plate number, is securely stored on the blockchain. Any user should be able to retrieve the image and its corresponding license plate number with ease by simply making a graphQL query to the notice list generated by the DApp.
-
Milestone 3: Provide comprehensive documentation. We are committed to providing extensive documentation and open source code for the image recognition system. This will empower developers and the community to understand and utilize the technology for various applications.
How you will use Cartesi, specifically?
By leveraging Cartesi’s innovative Virtual Machine, the process of identifying the number plate of violating vehicles through AI models reaches new heights of efficiency and security. By executing the AI model within the Cartesi Virtual Machine, we ensure swift and accurate processing of data, enabling seamless identification of number plates.
One of the most significant advantages of employing Cartesi is its integration with the blockchain, where the results generated by the AI model are securely stored. This integration creates a tamper-resistant and unalterable record, establishing a level of transparency and trustworthiness that traditional systems cannot match. Embracing Cartesi technology empowers us to maintain a distinct and immutable trail of information, paving the way for enhanced accountability and reliability in the identification and management of violating vehicles.
The following diagram shows the workflow from the upload of the image to the application until the infraction is reported by the user.
-
The user uploads a photo of the vehicle in infraction to the App.
-
In the App the photo is transformed into bytes.
-
The encoded image is sent as input to the DApp via a payload.
-
The input data is analyzed through the execution of Python scripts that retrieve the image and process it to obtain the bounding box containing the license plate (object detection algorithm, YOLO) and the characters contained in it (character recognition algorithm, OCR).
-
The Notice generated by the DApp is obtained from the Cartesi node.
-
The byte-encoded image (or it’s hash) and the license plate number obtained are returned to the app.
-
The infraction notification is sent to the authority and, if possible, to the infractor.
-
The infractor can verify the computation and present a fraud proof if he/she found flaws. If the infractor doesn’t complain within a given challenging period (Cartesi’s dApp epoch), the computation is confirmed on-chain. In case the result is disputed, the Cartesi’s interactive dispute resolution system hops in and ensures the correct outcome.
-
Once the final results are obtained, the Notice is confirmed on-chain.
At a future stage, a voucher could be created that adds the user’s wallet to a smart contract, ensuring that, when the offender pays, payment is received directly by the complainant with an on-chain protocol. Using cartesi vouchers, it could be possible to interact with base chain to ensure that the payment flow of the fine is also recorded on-chain.
Technical details
The license plate recognition model will be built by harnessing the synergistic potential of OpenCV, YOLO, and Optical Character Recognition (OCR). OpenCV, a versatile open-source library for computer vision and image processing, will empower the model with efficient image manipulation, key feature extraction, and vital preprocessing capabilities. YOLO (You Only Look Once), a cutting-edge object detection framework, renowned for real-time precision, will swiftly identify license plates in images. The integration of OCR will further enhance the model’s ability to accurately recognize and decipher characters on license plates, ensuring comprehensive and swift results.
Value Proposition
The successful execution of the POC can pique the interest of the community and other developers for the following reasons:
-
Accompanying AI progress: In light of the swift and constantly evolving progress of artificial intelligence models, it becomes increasingly crucial to demonstrate Cartesi’s ability to harness and leverage these cutting-edge technologies. By showcasing Cartesi’s potential to integrate with new and revolutionary AI advancements, we not only inspire the community but also underscore the platform’s capability to drive innovation and stay at the forefront of transformative developments.
-
Showcasing Real-World Use Cases: By addressing a practical problem in traffic management and law enforcement, the project showcases the versatility and relevance of Cartesi’s technology beyond theoretical concepts.
-
Community Engagement and Adoption: The application’s citizen reporting feature fosters community engagement and encourages public participation in addressing traffic violations. Developers interested in building applications with community-driven components may find inspiration from this aspect of the project.
Estimated Duration and Funds Requested
Duration: 1 month
Total Funds request: $5000
-
Milestone 1: 1500
-
Milestone 2: 1500
-
Milestone 3: 2000
Subsequent Vision and Extensibility
If the proof of concept is successful, it will be possible to move forward with developing and implementing a large-scale project to detect vehicle violations. This project aims to transform traffic management by offering an effective, transparent, and secure solution for both citizens and authorities. The following steps will be taken to achieve this goal.
Video-Based Detection
In this step, the application extends its capabilities to process recorded video footage for infraction detection. The algorithm developed in the previous step is adapted to analyze video frames systematically and identify license plate violations within the recorded content.
Infraction Detection
Building upon the achievements of the POC, the next crucial stage focuses on enhancing the application’s capabilities to detect violations from static images. Advanced computer vision techniques and efficient processing algorithms are implemented to analyze images and identify vehicles with infractions accurately.
Integration of Infraction Database
In the third step, the application incorporates an infraction database containing a list of license plates associated with arrest warrants or other legal actions. Whenever a license plate match is found between the detected vehicles and the entries in the infraction database, appropriate alerts or notifications are triggered.
User-Friendly Interface and Rewards
Moreover, we propose that users who provide convincing evidence leading to the identification of offending vehicles can be rewarded with a portion of the fine imposed for the offense. This innovative approach not only encourages community participation but also establishes a collaborative and community-driven framework for traffic enforcement.
Reusability and Other Use Cases
The reusability of this vehicle license plate recognition program offers a wide range of possibilities for applications in different contexts and sectors. Thanks to its modular architecture and the use of open-source technologies, the program can be adapted and reused to address various issues in today’s society. Some prominent use cases include:
-
Traffic Management: In addition to detecting traffic violations, the system can be used to monitor and improve traffic flow in urban areas and highways. Identifying illegally parked vehicles, analyzing traffic patterns, and measuring vehicle flow are some examples of how this solution can contribute to more efficient and safer traffic management.
-
Public Safety: Law enforcement agencies can utilize the program to identify vehicles involved in criminal activities or with pending arrest warrants. By integrating the infringement database with police records, surveillance and response to criminal incidents can be enhanced, increasing safety in communities.
-
Parking Lots and Tolls: The system can be applied in public and private parking lots to facilitate vehicle entry and exit through automatic license plate recognition. Additionally, in road toll systems, it can be used to expedite fee collection without the need for vehicle stops.
-
Logistics and Fleet Management: In the logistics and transportation sector, the program can be used to track and manage vehicle fleets, ensuring more effective control over the delivery of goods and itinerary tracking.
-
Residential Parking Security: In residential areas, the system can be employed to detect unauthorized vehicles parked in private spaces, providing better management and security for residents.
-
Environmental Surveillance: By integrating surveillance cameras with the system, it becomes possible to identify polluting vehicles or those in violation of environmental regulations, facilitating the implementation of measures to protect the environment.
-
Industrial Automation: In the industry, the program can be used to identify and track vehicles within industrial complexes, enhancing logistical management and security in these areas.
Risks and Contingency Plans
If the license plate detection algorithm’s performance and accuracy fall below expectations, we may need to explore alternative AI techniques or models. In some cases, we may need to implement an algorithm to improve the quality of the images for more accurate recognition of the number plate.
Success criteria.
In order for the proof of concept to be considered a success, the following conditions must be fulfilled:
-
The model running on the Cartesi Virtual Machine must achieve a high level of accuracy in identifying license plates from images, exceeding a certain pre-set threshold.
-
The results obtained must be successfully stored on-chain.
Final report
The Think and Dev team is committed to give this code to the community. In case we fail or accomplish the goals, we leave the project open to whoever wants to take it up, fork it, complement it, adapt it, and evolve it.
About Your Development Team
- Lucas Marc
Blockchain Technical Leader, working on Think and Dev leading multidisciplinary teams in different verticals.
- Juan Manuel Aragón Paz Ph.D
Ph.D. in Geophysics. Knowledge of GNSS processing, Python, and data science and analysis. Graduate of the National University of La Plata. More than 10 years as a teaching assistant in different subjects. His first WP was published last April.
- María Mercedes Vazzano Ph.D
PhD in Astronomy. Experience in the use of Python libraries for data management and visualization tools, as well as in the implementation of computer vision models. Graduate of the National University of La Plata.
ERC-20 Payee address
0x5d7224E3A04384c44E2De3ed5D135181d0f3a87a