AI-Powered System for Detecting License Plate Violations on Vehicles

POC AI-Powered System for Detecting License Plate Violations on Vehicles.

The following proposal is submitted on behalf of Think and Dev, a pioneering technology company focused on blockchain innovation. We are seeking funding to develop an advanced image recognition system with artificial intelligence (AI) capabilities, integrated into Cartesi’s infrastructure.

Core Concept and purpose statement

Project Description

The proposed project aims to build an on-chain-verifiable image recognition system using AI algorithms, which can be run on the Cartesi Virtual Machine and whose results can be stored in the blockchain. The first use case of the system will be to identify number plates of vehicles in violation of the law. When a number plate is identified, it will be stored on-chain ensuring irrefutable proof of the offense.

The ultimate goal of this project is to provide full documentation, open-source code, and pre-built Wheels to encourage knowledge sharing and create an open-source repository. We are committed to delivering quality and well-documented code that leverages Cartesi’s capabilities to address real-life problems using cutting-edge technology. In addition, we will include pre-built Wheels to facilitate seamless installation and usage of our software by the community.

Also, as we advance with the project, we will be trying to generate examples, extract snippets of code and generate discussions on discord that could be useful for the community to understand different Cartesi use cases.

We will prove that we can improve traffic safety by using blockchain and IA to automate the detection of number plates and the identification of traffic violations.

Project Goals

  • Milestone 1: Create a Proof of Concept (PoC) demonstrating the capabilities of our AI-based traffic violation recognition system when executed within the Cartesi Machine. This PoC will showcase the power of Cartesi by running a trained model (using the YOLO object detection algorithm and character recognition systems) that can accurately identify license plates in images of vehicles involved in traffic offenses. The focus will be on the practical application of Cartesi for real-time predictions, excluding the resource-intensive model training phase.

  • Milestone 2: Build a user-friendly interface that seamlessly integrates image input into the Cartesi Machine. This interface is vital for ensuring a smooth and secure process of processing and storing images, along with their associated license plate numbers, on the blockchain. We will ensure that anyone can easily retrieve this information by making a simple GraphQL query to the notice list generated by our Decentralized Application (DApp). This milestone will serve as a practical example of Cartesi’s utility in enhancing data accessibility and management.

  • Milestone 3: Foster community learning and engagement by providing comprehensive documentation and open-source code for the image recognition system. Our commitment to transparent documentation will empower developers and enthusiasts within the Cartesi community to comprehend and utilize the technology for diverse applications. By sharing our findings and experiences openly, we aim to catalyze discussions and knowledge-sharing within the Cartesi ecosystem, showcasing its potential for a wide range of projects beyond our traffic violation recognition system.

How you will use Cartesi, specifically?

By leveraging Cartesi’s innovative Virtual Machine, the process of identifying the number plate of violating vehicles through AI models reaches new heights of efficiency and security. By executing the AI model within the Cartesi Virtual Machine, we ensure swift and accurate processing of data, enabling seamless identification of number plates.

One of the most significant advantages of employing Cartesi is its integration with the blockchain, where the results generated by the AI model are securely stored. This integration creates a tamper-resistant and unalterable record, establishing a level of transparency and trustworthiness that traditional systems cannot match. Embracing Cartesi technology empowers us to maintain a distinct and immutable trail of information, paving the way for enhanced accountability and reliability in the identification and management of violating vehicles.

The following diagram shows the workflow from the upload of the image to the application until the infraction is reported by the user.

  1. The user uploads a photo of the vehicle in infraction to the App.

  2. In the App the photo is transformed into bytes.

  3. The encoded image is sent as input to the DApp via a payload.

  4. The input data is analyzed through the execution of Python scripts that retrieve the image and process it to obtain the bounding box containing the license plate (object detection algorithm, YOLO) and the characters contained in it (character recognition algorithm, OCR).

  5. The Notice generated by the DApp is obtained from the Cartesi node.

  6. The byte-encoded image (or it’s hash) and the license plate number obtained are returned to the app.

  7. The infraction notification is sent to the authority and, if possible, to the infractor.

  8. The infractor can verify the computation and present a fraud proof if he/she found flaws. If the infractor doesn’t complain within a given challenging period (Cartesi’s dApp epoch), the computation is confirmed on-chain. In case the result is disputed, the Cartesi’s interactive dispute resolution system hops in and ensures the correct outcome.

  9. Once the final results are obtained, the Notice is confirmed on-chain.

At a future stage, a voucher could be created that adds the user’s wallet to a smart contract, ensuring that, when the offender pays, payment is received directly by the complainant with an on-chain protocol. Using cartesi vouchers, it could be possible to interact with base chain to ensure that the payment flow of the fine is also recorded on-chain.

Technical details

The license plate recognition model will be built by harnessing the synergistic potential of OpenCV, YOLO, and Optical Character Recognition (OCR). OpenCV, a versatile open-source library for computer vision and image processing, will empower the model with efficient image manipulation, key feature extraction, and vital preprocessing capabilities. YOLO (You Only Look Once), a cutting-edge object detection framework, renowned for real-time precision, will swiftly identify license plates in images. The integration of OCR will further enhance the model’s ability to accurately recognize and decipher characters on license plates, ensuring comprehensive and swift results.

Value Proposition

The successful execution of the POC can pique the interest of the community and other developers for the following reasons:

  • Accompanying AI progress: In light of the swift and constantly evolving progress of artificial intelligence models, it becomes increasingly crucial to demonstrate Cartesi’s ability to harness and leverage these cutting-edge technologies. By showcasing Cartesi’s potential to integrate with new and revolutionary AI advancements, we not only inspire the community but also underscore the platform’s capability to drive innovation and stay at the forefront of transformative developments.

  • Showcasing Real-World Use Cases: By addressing a practical problem in traffic management and law enforcement, the project showcases the versatility and relevance of Cartesi’s technology beyond theoretical concepts.

  • Community Engagement and Adoption: The application’s citizen reporting feature fosters community engagement and encourages public participation in addressing traffic violations. Developers interested in building applications with community-driven components may find inspiration from this aspect of the project.

Estimated Duration and Funds Requested

Duration: 1 month

Total Funds request: $5000

  • Milestone 1: 1500

  • Milestone 2: 1500

  • Milestone 3: 2000

Subsequent Vision and Extensibility

If the proof of concept is successful, it will be possible to move forward with developing and implementing a large-scale project to detect vehicle violations. This project aims to transform traffic management by offering an effective, transparent, and secure solution for both citizens and authorities. The following steps will be taken to achieve this goal.

Video-Based Detection

In this step, the application extends its capabilities to process recorded video footage for infraction detection. The algorithm developed in the previous step is adapted to analyze video frames systematically and identify license plate violations within the recorded content.

Infraction Detection

Building upon the achievements of the POC, the next crucial stage focuses on enhancing the application’s capabilities to detect violations from static images. Advanced computer vision techniques and efficient processing algorithms are implemented to analyze images and identify vehicles with infractions accurately.

Integration of Infraction Database

In the third step, the application incorporates an infraction database containing a list of license plates associated with arrest warrants or other legal actions. Whenever a license plate match is found between the detected vehicles and the entries in the infraction database, appropriate alerts or notifications are triggered.

User-Friendly Interface and Rewards

Moreover, we propose that users who provide convincing evidence leading to the identification of offending vehicles can be rewarded with a portion of the fine imposed for the offense. This innovative approach not only encourages community participation but also establishes a collaborative and community-driven framework for traffic enforcement.

Reusability and Other Use Cases

The reusability of this vehicle license plate recognition program offers a wide range of possibilities for applications in different contexts and sectors. Thanks to its modular architecture and the use of open-source technologies, the program can be adapted and reused to address various issues in today’s society. Some prominent use cases include:

  • Traffic Management: In addition to detecting traffic violations, the system can be used to monitor and improve traffic flow in urban areas and highways. Identifying illegally parked vehicles, analyzing traffic patterns, and measuring vehicle flow are some examples of how this solution can contribute to more efficient and safer traffic management.

  • Public Safety: Law enforcement agencies can utilize the program to identify vehicles involved in criminal activities or with pending arrest warrants. By integrating the infringement database with police records, surveillance and response to criminal incidents can be enhanced, increasing safety in communities.

  • Parking Lots and Tolls: The system can be applied in public and private parking lots to facilitate vehicle entry and exit through automatic license plate recognition. Additionally, in road toll systems, it can be used to expedite fee collection without the need for vehicle stops.

  • Logistics and Fleet Management: In the logistics and transportation sector, the program can be used to track and manage vehicle fleets, ensuring more effective control over the delivery of goods and itinerary tracking.

  • Residential Parking Security: In residential areas, the system can be employed to detect unauthorized vehicles parked in private spaces, providing better management and security for residents.

  • Environmental Surveillance: By integrating surveillance cameras with the system, it becomes possible to identify polluting vehicles or those in violation of environmental regulations, facilitating the implementation of measures to protect the environment.

  • Industrial Automation: In the industry, the program can be used to identify and track vehicles within industrial complexes, enhancing logistical management and security in these areas.

Risks and Contingency Plans

If the license plate detection algorithm’s performance and accuracy fall below expectations, we may need to explore alternative AI techniques or models. In some cases, we may need to implement an algorithm to improve the quality of the images for more accurate recognition of the number plate.

Success criteria.

In order for the proof of concept to be considered a success, the following conditions must be fulfilled:

  • The model running on the Cartesi Virtual Machine must achieve a high level of accuracy in identifying license plates from images, exceeding a certain pre-set threshold.

  • The results obtained must be successfully stored on-chain.

Final report

The Think and Dev team is committed to give this code to the community. In case we fail or accomplish the goals, we leave the project open to whoever wants to take it up, fork it, complement it, adapt it, and evolve it.

About Your Development Team

  1. Lucas Marc

Blockchain Technical Leader, working on Think and Dev leading multidisciplinary teams in different verticals.

  1. Juan Manuel Aragón Paz Ph.D

Ph.D. in Geophysics. Knowledge of GNSS processing, Python, and data science and analysis. Graduate of the National University of La Plata. More than 10 years as a teaching assistant in different subjects. His first WP was published last April.

  1. María Mercedes Vazzano Ph.D

PhD in Astronomy. Experience in the use of Python libraries for data management and visualization tools, as well as in the implementation of computer vision models. Graduate of the National University of La Plata.

ERC-20 Payee address



Thanks for your proposal, and for giving a complete response to all sections of the template. Indeed, automatic license plate recognition is an interesting problem. However, I feel that I speak for the rest of the grant committee when I say that there are many details that you’ve left undiscussed in the “Why Cartesi” section.

Would you mind putting together a diagram and technical explanation that details how your system will operate inside of the Cartesi Virtual Machine?

Hi ThinkandDev!

As a DevAd, I really liked your proposal. I think it can have a positive impact on the developer community. Below, I’ve compiled a few points to help improve it:

  • Making libs and useful tooling that you port or create separate deliverables would be valuable additions to your proposal. For instance, Felipe Grael from the Prototype Unit has been maintaining a repository of python wheels that are fully compatible with the CM RiscV’s implementation: Wheel Package Index.
  • It would also be nice if you could participate in Discord calls or write a few articles sharing your experience with the community.

  • Just emphasizing what Joe mentioned, in the spirit of being an accessible reading for everyone in the community, some high-level diagrams detailing how things would work would be a real nice addition.

1 Like

Hey ThinkandDev! Interesting proposal. Would be curious to dig a bit deeper on a basic threshold question here: why blockchain?

For example, there are already plenty of automated, camera-based traffic systems that capture images of speeding vehicles, cross-reference the license against a registration database, and mail tickets to the vehicle’s owner.

I’ve received a few speeding tickets from cameras myself!

A time-stamped digital image is pretty solid proof of the infraction. And, as far as I know, there aren’t any widespread failures or inaccuracies in this process that instill any serious lack of trust in these centralized systems as they are.

And so, in your view, what incremental value does putting this process on blockchain add, particularly when compared to the additional costs associated with processing and storing all of this data on-chain?

1 Like

Thank you @joe-cartesi and @gabrielbarros for your contributions. We have updated the proposal including your suggestions.

1 Like

Very good question @Brandon!

In the future, we want to connect the available cameras (Ford SDK, Tesla API, etc.) of all cars driving on the street, and have your own car detecting and impacting on the blockchain an unalterable proof of the infraction.

The Web2 solutions may be alterable by the user or the issuer. Particularly, in LATAM there are many cases of municipalities that erase infractions after receiving a bribe. With Web3, there remains a public proof of each infraction, which anyone can read and check.

Additionally, this approach encourages users to act as agents by providing them with a percentage of the fine. It also deters potential violators due to the possibility of being reported by any citizen. This decentralization approach can reduce the expenses associated with hiring traffic agents.


Hey @Brandon full verifiability, transparency, immutability and better incentives alignment between population and public authorities is the reason I see this is an interesting use cases, as well many variants around the same topic.
Technically I see data availability a challenge to overcome since data on blockchain, especially if we talk about ETH mainnet, is hell of expensive. A solution for that would be use IoT related cheaper chains such as IoTEX where we also have our infrastructure deployed.

The importance here is to create a first PoC to test the waters and fully map the technical challenges in order to later progress toward a more robust application with more functionalities, including a more sophisticated economy (allowing better aligments of incentives

1 Like

Just curious, but how would the system you’re proposing handle false positives or ghost plates? My current understanding is license plate reader accuracy is not 100%. If I’m wrong, please correct me. Is this an appeal process that would be handled by local jurisdictions outside of this proposed system?

1 Like

Great description! Well thought out and well explained.

Technically, I see a lot of issues with this proposal which I believe are insurmountable. I’ll categorize in issues specific to this solution, and in issues in general with this approach.

Specific issues

  • Recognizing license plates is a completely different problem than recognizing infractions. I’m extremely skeptical the latter is doable with current technology.

  • Users that submit pictures can always lie. They could fake an image of license plate of their annoying neighbor and post it to the system. They could fake it digitally, or even physically; it’s easy to create a picture of any license plate.

  • Current solutions work pretty well. Furthermore, this proposal is already centralized: it needs government coercion. At this point, why a blockchain? The issues described in the replies are better addressed with a web2 solution.

Issues general with AI and image recognition

  • AI can always be fooled given arbitrary inputs. It’s an arms race: the more sophisticated models, the more sophisticated foolers. It’s possible to generate images with seemingly random noise/static that hit false positives. A dishonest user can benefit a lot from hacking the system, so expect it to be hacked this way.

  • Putting images onchain is not currently feasible.

1 Like

Hi @GCdePaula!

We acknowledge the challenges presented in the proposal, but do not consider the issues to be insurmountable. Here is our explanation for the issues you have brought up.

Specific issues

  • As part of our proof of concept, we propose the recognition of number plates of vehicles that are known to be in violation of the law. We are aware of the complications that could be involved in carrying out the recognition of infringements and for that reason this is proposed as a possible future work.
  • Our proposal currently focuses on implementing plate number recognition in user-uploaded photos as a proof of concept. The use of doctored images falls outside the scope of the POC. In the future, other mechanisms could be implemented to prevent tampering, not necessarily based on Web3. Therefore, in this instance we only intend to use on-chain verification to confirm whether an image contains a patent or not, even if it has been manipulated.
  • As for Why Blockchain, firstly, this is a case study to see the viability of the technology in the short and long term. Secondly, transparency. As we answered in a previous comment, Web2 solutions can be alterable by the user or the issuer. With Web3, there remains a public proof of each infraction, which anyone can read and check.

Issues general with AI and image recognition

  • We understand that AI can be tricked and false positives can be generated. For the moment we are proposing a proof of concept to assess feasibility. In the long term, the model needs to be much better trained and more controlled. The proposal is that the system will help people, but it will not work in an unsupervised way until we are absolutely sure that it works.
  • Our intention is to store the hash of the image, not the image itself

Our main goal is to provide the whole community with open source code that demonstrates how to use these libraries to simplify the use of these technologies for this case or any similar ones.

1 Like

Hi guys!
We appreciate the interest in our proposal and the feedback received. If there are no further questions or suggestions, we believe we can label the proposal as REVIEW. Is that correct, @joe-cartesi?

That is correct. From this point onward, however, @hellen will be the CGP’s new administrator. Please reach out to her to coordinate a final review from Cartesi’s technical minds before it is decided whether it should go to a vote. All the best!

Hello @ThinkandDev , I’ll be reaching out soon via dms on the forum to initiate a discussion on the next steps.

1 Like

Ey @hellen , thanks for taking the time to meet with us! We’ve have updated the milestones section according to the feedback you guys gave us. Let us know if it’s ok now, or if there’s something else that we should modify.

This proposal is now live for voting on snapshot.

Congratulations @ThinkandDev. Your proposal has passed! Huge thank you to everyone who contributed and shared their insights. The Grants Council will be in touch with you to discuss next steps and offer support and guidance on your project.

Edit: This project has been moved to the “Funded projects” category.

1 Like

This project has been completed

Summary: The AI License plate violation detector (Whistleblower) Dapp is a revolutionary application leveraging chain-verifiable image recognition powered by AI algorithms. Operating seamlessly within the Cartesi Machine, this innovative system specializes in identifying license plates associated with legal violations. By securely storing collected data, including images and license plate information, on the blockchain, the application establishes irrefutable evidence of traffic infractions.

Video Link:

GitHub Links:

Community Participation: We encourage the Cartesi Development Community to engage with this project:

1 Like