What we actually do?

Company introduction

Recon-ai is a young software company with four people currently working with us, and we are implementing artificial intelligence for machine vision. We are looking for projects and partners. As a software company, we are especially looking for partnerships with hardware manufacturers so that we can get the correct sensors for our customers as well as software partnerships to streamline our implementation process.

Why AI?

So what is the motivation to use artificial intelligence? Compared to traditional techniques, when properly trained, an AI solution can be computationally extremely lightweight while producing high quality results. In other words, AI is very cost-effective enabling real time on-site operation on embedded systems.

Chess example

Here is a fun little example: Stock fish 8 is a chess engine that has been superior to human players for many years now. It considers 70 million move possibilities per second and is taught with basically the entire history of professional chess matches. Google’s Alphazero is a “true” AI that considers only 80 000 move possibilities per second, only played against itself for a few hours, and beat the Stockfish 8 in 27 games out of one hundred and never lost. Now of course, this type of AI is quite different than what we are (at least currently) using, but it highlights the advantages of AI well.

Screen Shot 2018-09-28 at 9.08.23.png

Needs

What does an AI need for the lightweight operation? The computationally heavy part lies in the training, for which the AI needs to process a lot of data. However, this, in addition to testing and validation, can be done in advance on for example some computer cluster. The network itself is light, and can be then transferred easily to the on-site machine via for example the mobile network. There is also the added benefit of the automatization of updating the AI for additional functionalities or fidelity as per the client’s needs, which is convenient.

Screen Shot 2018-09-28 at 8.47.45.png

Training the AI

A bit about the training process. AI is essentially a combination of a network architecture and network weights. The architecture must be designed for a given “problem type”, and the weights are then optimized for the problem which is called “training.” So first the network “architecture” must be designed. After that, the training data must be gathered and labeled which means essentially solving the problem for that data, for example, if AI is supposed to find certain points, the points must be marked on the training data. This labeled data can then be augmented, which is essentially using certain tricks to multiply the data for training. Finally, the augmented and labeled data is ran through the untrained network and the weights are optimized by a certain algorithm. The training process gives us the weights which in conjunction with the architecture form the AI that can now be used.

AM-X

Now, what we are actually offering is a platform that can be utilized to implement AI based solution to various different machine-vision related problems. As I said before, the solutions are expected to be relatively cost-effective, computationally lightweight, convenient to use and to update or expand upon.

What we need from the client is obviously the description of the problem and a large set of representative data. Based on the possible existing sensors and the problem, we can assess whether or how easily the problem can be solved. If there are no existing sources of data, we will assess the needs based on the problem description. As a software company we rely on partnerships with hardware manufacturers to provide the required sensors. It is important to note that although we can certainly assess the suitability of particular type of data, the AI needs to be trained before using or testing it.

Screen Shot 2018-09-28 at 8.48.55.png

Here is an illustration of how the relationship between us and the client could work. As I mentioned, we need the data and the problem description. The sensors, i.e., the source of the data can be pre-existing or new. We will of course help with the choice of sensors should new hardware be needed.  Based on the data and the problem description, we will build the AI, train it, test it and implement it for the on-site application. As the implementation is operational, new data can be gathered for additional functionalities or further improvement of accuracy should the conditions for example change.

AM-RAIL™️ example

Railinfra

Here is an example for the railroad industry. In order for the trains to run safely and on time, the surrounding infrastructure has to work really well. However, the different components of the infrastructure of for example the electrification and the superstructure wear down and may eventually break. If it goes to that point, the economical and even human damages can be quite serious. This is why preventive maintenance and asset management are so important so that it never gets to that point. Thus, extensive active monitoring is required. Many of the current monitoring equipment rely on these heavy measurement wagons that are expensive to buy, run and maintain. This is where we come in.

information

Our AM-Rail platform is based on relatively cheap sensors that can be installed on for example cargo trains without the need for an external wagon. We then use artificial intelligence to train and analyze the data feed from the sensors, i.e., cameras, lidars, etc., and extract the relevant information from the feed. The data analysis can be done conveniently with a light and cheap embedded device on board of the train in real time. And like I said before, the software can be updated easily with the mobile network. Thus, money is saved.

Measure

Here is an example of implementation. The point here is to measure the contact point height with respect to the rail level and its horizontal displacement from the center of the rails. The contact point is located at the top of the red line in the picture. This image is a frame from a video recorded with a single consumer grade camera. First, the AI finds the rails and three points of interest from the frame. The points of interest are the previously mentioned contact point (top of the vertical red line) and the two end points of the pole (endpoints of the blue line). Since this is done with a single camera, the pole is needed for reference as the picture doesn’t contain depth information. After the points and rails are found, the two measures can be calculated using simple math. Here, we reach approximately 3% accuracy with a consumer grade camera and an AI that runs on a cheap CPU and whose training cost next to nothing.

Next, we plan to implement a lidar system in conjunction with the camera in order to get rid of the referencing and increase the accuracy, which for this particular application is important. With the lidar providing the depth information, we can fully map the 3D location of the points of interest with respect to the sensor. We are also planning to introduce detection and measurement of additional components to the solution.

Our overarching plan is to move towards building information modeling by 3D object mapping, detection, orientation and/or location with respect to object information for example as CAD files for different ecosystems. Multiple already existing types of sensors could be utilized such as CCTV cameras, already existing cameras in buses or trams, and so on. A larger model could then be constructed based on the large data set from a variety of sensors located all around the place.

Conclusion

So, in summary, utilizing AI for machine vision can offer high quality information on site and real time with very competitive costs. And we offer such a platform which can be used for AI based recognition, measurement and locating of objects based on optical data. We are currently looking for a pilot project as well as partnerships with hardware manufacturers in order to streamline the platform for clients without proper already existing hardware.

So if you are interested, do not hesitate to contact us! Thank you!




Kalle Koskinen

CTO

Recon AI

Implementation of AM-RAIL™


 

CONTENT

Foreword

Implementation

Ecosystem - Added value

Pilot?

Foreword

In this article, we describe how the AM-RAIL™ can be implemented to our clients' ecosystems. The article includes a description of the implementation process and explains the added value to a railway operations. The goal of this article is to give an idea of how AM-RAIL™ can be implemented to our clients' ecosystem, as we aim to gain pilot projects for AM-RAIL™. 

Implementation

Implementing AM-RAIL™ does not require expertise of AI-solutions from the client. Only data from the client's ecosystem is required. This data can be gathered from existing or newly installed sensors (Cameras, LiDars, Sonars). The data gathered from the field needs to be related to a CAD drawing of the components. These CAD drawings can also be used for recognition operations by rendering the drawings and relating them to the sensor data.

Implementation process

Phase 1 - Data acquisition. The client provides the CAD drawings, video and LiDar cloud data from the railway network. This can be done by using existing data or by gathering data with a new hardware set-up. 

Phase 2 - Data annotation. Recon AI labels the video and LiDar data and renders the CAD drawings to annotate the data for the training phase. 

Phase 3 - Training. Recon AI plans, builds and tests neural network models for training the AI to gain sufficient recognition performance.

Phase 4 - Deployment. Recon AI deploys the trained AI algorithms to the on-site sensors via API.

Phase 5 - Development. Recon AI further develops the solution in order to recognise additional components of interest and to improve recognition performance and measurement accuracy.

 We estimate the first implementation circle to take 6-12 months.

 

Ecosystem - Added Value

Immediate added value of the AM-RAIL™ is the automation of measurements performed currently in installation operations. In addition to this, point-cloud information of components can be utilized for 3D modelling of infrastructure (asset management) and combining information of infrastructural changes for predictive maintenance operations.

Screen Shot 2018-09-26 at 9.40.32.png

Pilot?

We are looking for clients to start a pilot with. Interested? Feel free to contact:

info@recon-ai.fi

+358 (0) 50 587 1254

More information:

Article: AM-RAIL™ AS A PART OF SMART CITY

Video:

Showing is better than telling - take a look at the AM-City™ video.

Recon AI
Henri Memonen

 

 

AM-RAIL™ as a part of Smart City


 

CONTENT

Foreword

What is a Smart City?

Ecosystem

Implementation

Summary of benefits to different stakeholders

Emphasize on data-acquisition on an early stage

Foreword

In this article, we describe the use case of AM-X™ platform in Smart City transportation, the requirements for the ecosystem of the client, and implementation of the solution emphasizing the benefits in both short- and long-term. The goal of this article is to give an idea of how AI-solutions can be implemented to our clients' ecosystem and to emphasize the importance of gathering vast amounts of clear data at an early stage in the development. 

What is a Smart City?

”A smart city is an urban area that uses different types of electronic data collection sensors to supply information which is used to manage assets and resources efficiently. This includes data collected from citizens, devices, and assets that is processed and analyzed to monitor and manage traffic and transportation systems, power plants, water supply networks, waste management, law enforcement, information systems, schools, libraries, hospitals, and other community services.”

*https://en.wikipedia.org/wiki/Smart_city 

Ecosystem

The ecosystem is commonly built on the following three essential elements:

On a common level, the process of these kinds of ecosystems are quite straightforward.

  1. Sensors measure the environment and collect data for analytic platforms, with time and location labels.

  2. Analytics platforms provide analyzed guidance information to the client’s ERP-system.

  3. On ERP-system, operators are inspecting and labeling the data, and the labels are provided back to the analytics platform.

  4. Self-learning algorithms on the analytics platform read measurement data to provide more accurate predictions per each new data-point.

Screen Shot 2018-04-10 at 14.10.44.png

When implementing the AM-RAIL™ as a part of Smart City, a variety of options for the sensors, analytic platform models and ERP-systems are available for the ecosystem. Some of the options and their related considerations are listed below as suggestions that we recognize as valuable to the Smart City ecosystem.

Sensors:

All sensor data should be labeled with time and location labels when using the sensor.

 Analytic platform models:

ERP-systems:

We have categorized the labels into the following categories:

  • Fault labels (a label that indicates a fault in the ecosystem)

  • Effect labels (a label that indicates causal effect in the ecosystem)

  • Component labels (data that is labeled relation to a component)

  • Simulated data labels (simulated data label from engineering models)

Implementation
 

We often hear from our networks that they do not wish to be first ones to implement AI-solutions to their own operations, but want to wait for markets to mature and consider applying AI-solutions at a later stage. We believe that the major fault in this reasoning is that after AI-solutions have developed enough, the AI would be easily applicable to various operation models. Unfortunately, this mix-up of the intelligence explosion and use-case based machine learning application is a common mistake. As intelligence explosion would make a machine that only requires calculation power to develop, machine learning requires adaptation to the specific user cases. If the assumption of the intelligence explosion is faulty, it does not matter, because the machine would still be solving most of our problems, one way or another.

The second line of reasoning we often hear from our clients is that the system would only bring benefits after several years of training of the AI, which would mean that ROI for the project investment does not make sense. Our belief is that when the implementation process is built smartly, we can add value to the clients´ ecosystem early on in different phases that support each other. This way the investment will start paying back quickly, and the benefits of the system will increase when more data is gathered.

On implementing AM-RAIL™ to railway and city maintenance operations, we estimated the following timetable and benefits from the system.

Screen Shot 2018-04-11 at 9.30.24.png

Phase 1, Sensors and improvement of the current planned predictive maintenance by providing an increasing amount of data from the field. This data can be used as an alarm-rate maintenance operation right after the sensor data is gathered. The data is stored in the database. The estimated time for implementation is 1,5 – 2 years. Components can be recognized and the information can be used for ownership management.

Phase 2, An analytics platform is chosen and implemented. The analytical platform can recognize decay rates from the data gathered from the sensor. The decay information can be used for maintenance operations and the operations can be moved towards a preventive maintenance model.

Phase 3, The analytics platform starts to receive labels from the ERP-systems for self-learning algorithms. The self-learning algorithms will receive fault, effect, component and simulated data labels from ERP-systems which leads to the platform performing better with each new data point. Maintenance can be moved towards a predictive maintenance model.

Phase 4, AR-solutions can be implemented to operations to provide guidance for the field operators. While providing guidance the guiding person can label components from the pictures and train machine vision solutions for recognition of these components. After a sufficient amount of component labels, machine vision can be used for guiding robotics and maintenance the operations can be further automated in steps. The ultimate goal is set to automate maintenance operations fully.

As we see, AI-solutions will bring benefits to operations also in short-term.

 

A summary of the benefits to different stakeholders

Construction and maintenance operators

Recognizing the need for maintenance before additional cost occurs - moving to a Predictive maintenance model. This increases route capacity and decreases costs of maintenance operations by decreasing down-time of the track. In addition, component recognition can be used to maintain uptodate information for ownership management.

By adding measurement solutions to provide data for a BIM engineering software and providing real-time uptodate information, the right baseline information can be provided for designing operations. These measurement solutions will also remove the need for manual measurement of structures and gathering the measurement data into database can be automated.

In addition, an AR-platform can be used to increase efficiency and quality in the installation operations by providing guidance remote guidance during installation operations. At the same time, operators can label videodata and labels can be used for the training of machine vision solutions.

For traffic and transportation operators

Moving into a Predictive maintenance model decreases requirement for maintenance operations on the route and improves route capacity.  Facial recognition system provides smooth passenger transportation and provides baseline for TaaS-solutions.

Smart city, citizens of the city, Institutes and companies operating in the city

Camera systems attached to transportation machinery can measure and recognize components from surrounding structures in its route meaning anything from the surveillance of street-lights to the condition of buildings. Changes in these structures can be surveilled and data can be used for the allocation of maintenance resources. The same data can then be used for building a uptodate digital model of the city. All this data can be used as a platform for new inventions resulting in a better quality of life for the citizens.

 

Emphasis on data-acquisition on an early stage

Implementing AI-solutions to most of the ecosystem requires a lot of data to work with. The data needs to be raw, clear and labeled to provide sufficient information for self-learning algorithms that aim to provide information for operations and finally make automation possible for the application.

When a system requires huge amount of clean raw-data from several sensors, the makes cost of a single measurement to work as major cost-driver. This means heavy and expensive sensors wont do the trick, instead by using multiple cheap sensors in the ecosystem most of the benefits can be achieved.

In addition, the know-how of the field operators needs to be digitalised and stored as data. Machine learning means that the machine needs to learn what to do, and so far humans are acting as teachers. This means personnel working in the ecosystem needs to use digital tools on their operations in order to provide sufficient information for machines to eventually operate automatically.

We recommend all operators to start collecting data by implementing cheap sensors to their ecosystems and storing the raw-data to databases for the future’s needs. Ungathered information can never be recovered.

 

Recon AI
Henri Memonen

 

Don’t wait FOR your data to arrive -

let’s get it for you.