Traffic environment includes many different objects. For instance, traffic lights, signs, crosswalks, lines separating the lanes etc. It is a constant monitoring of the traffic conditions for the driver in order to obtain the necessary information for driving. Of course experience has important part in traffic when assessing the change in traffic lights or whether it is safe to pass the junction or not. Most of the traffic lights are programmed to change the light with certain delay, however some lights, like the ones used for crosswalks, have more flexible delay cycle that reacts to push buttons. Let us assume that Iseauto is able to obtain traffic light changing information via Internet before reaching the crossroad. This creates a situation where Iseauto could plan the route so that it reaches the crossroad exactly when the lights go green. In this way the driving becomes more steady that in turn makes the traffic more safer. This also adds the benefit of reducing energy consumption as less acceleration is needed. On the other hand, self-driving vehicles could transmit both location as well as route information to the traffic management system in order to control the traffic lights so that vehicles could pass more effectively. This is only but a single example for making traffic more safe as well as more energy  efficient by communicating with various traffic objects.

In addition to communication with static traffic objects, like traffic signs, lights etc, the self-driving vehicles (and perhaps also the man operated vehicles) could exchange information between themselves. Communication could take place in the same way as the popular navigation application Waze. Various traffic information is shared with other users, like dangers, accidents etc. Unlike humans the self-driving vehicle could transfer the traffic information into the centralized environment without ever losing track of the traffic thus reducing traffic hazards. For human it takes several seconds to note Waze about traffic situation, for the computer in self-driving vehicle only fraction of a second. Those couple of seconds, spent on making a note in Waze, could become crucial in order to avoid serious accident with casualties. Another important part of information sharing by self-driving vehicle is data gathered from on-board sensors. For example LIDAR outputs a local point cloud (simply put it is a local map) that could be uploaded to database for reuse by other self-driving vehicles in the future in the same area. This enables other self-driving vehicles to adjust quickly with the environment. As a conclusion the more self-driving vehicles in the traffic that share their local maps the bigger the area where all self-driving vehicles can manage more easily. Also more up to date information is available in the database. To sum up the previous discussion we can say that via communication between the self-driving vehicles both the traffic safety as well general management in difficult traffic situation could be improved.

When up until here we have described the activities that are directly linked with controlling self-driving vehicles, then having a direct Internet connection on board self-driving vehicles add some more benefits. For instance the vehicle could download and update it’s software autonomously. When a company hosts hundreds or even thousands of self-driving vehicles that take part in everyday traffic it is best that they run on the latest software (that is considered the best). In addition to self-driving software updates for example Iseauto could gain access to rector Jaak Aaviksoo personal calendar and be ready to transport him in time to some other campus building. We can also add communication with some household devices like coffee machine that receives information about homecoming time in order to make a fresh cup of coffee when stepping in the door. For the previous examples include both security as well as comfortability that self-driving vehicles could offer. Also the self-driving vehicle users are relieved from several common doings thus should have a rise in life quality.

Lastly, we should look at the communication between self-driving car and the user. In the previous chapter we already mentioned that self-driving vehicle could be in the right place by accessing information from the user’s calendar. However, peoples plans change often and not always the calendar does not have relevant information. Therefore the user could inform the vehicle to get him, regardless of the time. Nowadays, most of the workplaces are in downtown skyscrapers and often the there is not much space for parking and the cost for parking is high. So the self-driving vehicle could drive a bit away from the downtown after bringing the user to work and come back when the user send the request. Today this kind of information exchange with user is not feasible without Internet connection.

Everything we discussed is just a small example of the possibilities that self-driving vehicles presents as a part of the IoT. We can only wonder what kind of possibilities exist. The most important goals for self-driving vehicles in part of our everyday life are increasing the traffic safety and rising the life quality. As IoT and self-driving vehicles are part of rapidly developing field we can only wait what kind of interesting solutions are available for us in the future by the self-driving vehicles.

In the previous three sections we focused on developing the TalTech Iseauto, it is now time to take a look at what the final version of Iseauto currently looks like, which sensors are included in the bus, and what is it capable of.

Let’s start with the look of the bus, which is designed by Silberauto AS designer Sven Sellik. Iseauto is designed to be symmetrical, which makes it difficult to detect the rear and the front end of the bus by simply looking at it from the outside. The main color is dark gray, in addition, black and white colors are used. The height of the bus is 2.4 m, the length is 3.5 m and the width is 1.5 m and it weighs 1.1 tons. The following pictures give you some idea what a bus looks like.

The back of Iseauto[/caption]

Next, let’s look at where and how different sensors are located on the bus. First of all, the lidars, because their importance on the bus is difficult to overemphasize. We use Volodyne’s 3D lidars that create 360-degree 3D images of objects around the bus. They can be used for local positioning and object detection. Initially, it was planned to place one lidar on the roof at the front of the bus, and the other on the roof at the back of bus. But during the development, the results were achieved by placing two lidars in the front corners of the roof at a small angle (with an angle to the ground) and, in addition, one lidar was placed in the front of car a couple of hundred centimeters above the ground.

In addition to local positioning, the bus is able to position itself globally. For this purpose, the RTK-GNSS (Real Time Kinematic Global Navigation Satellite System) is placed on the bus, which is accurate to a few centimeters. Its main components are the GPS module and two antennas, one of which is positioned in front of the bus and the other at the rear. The use of two antennas gives you the opportunity to get the moving direction very quickly. The module itself was originally positioned at the back of bus above the engine, but because the model of RTK-GNSS used on Iseautol also has an IMU (inertial measurement unit), it needs to be repositioned because the high current moving in the engine produces a magnetic field, resulting in a magnetometer located in the IMU giving incorrect results. The IMU sensor includes three sensors – an accelerometer, a gyroscope and a magnetometer, or a compass, which in co-operation can output the bus trajectory. The picture below shows the antenna at the front of the bus.

The next important sensors are cameras. There is 5 cameras currently on the bus. Two cameras are looking to front of the bus – one on top and one on the bottom of the car. One camera looks behind the bus and two cameras to the sides (located in the front of the bus on the sides). The cameras can easily detect objects. Currently, Iseauto only identifies people and cars, but in the near future, for example, traffic signs and animals detection will also be added.

There are also 8 ultrasound sensors installed into the bus, which are designed to identify objects in the vicinity of the bus to a few meters. Four sensors were placed in front and four in back of the bus.

Now as we have described all of the senosors installed onto Iseauto, let’s look at what it can do.

Today’s main feature of Iseauto is that it can follow a given trajectory at a predetermined speed on a pre-mapped area. With the cameras, Iseauto can also identify people and vehicles in the area. The maximum permitted speed on the bus is 20 km / h. As of today, his main disadvantage is that it can’t identify objects on the trajectory lane and therefore, he also has no ability to cross over these objects, if necessary. However, these functionalities are in development. To ensure safety, there must be at least one operator during the bus ride to stop the bus from emergency brake button, if necessary. Emergency brake system was described in the previous section.

And finally, what’s the future of Iseauto? On November 14, 2018, two months after the first public demo, Jaak Aaviksoo, the TalTech Rector, and Väino Kaldoja, Chairman of the Silberauto AS Group, signed the agreement that will provide the basis for the continued development of Iseauto and the creation of a new Iseauto. The goal is to create an Iseauto2 that externally looks the same, but which would be both mechanically and technologically substantially optimized and designed to drive on the public streets. Its development should be completed by the summer of 2019. The first Iseauto, however, remains the development and training platform for TalTech students, and will be also the testbench for artificial intelligence and sensory testing for Iseauto2.

The Ministry of Education and Research and Estonian Research Council are supporting the completion of the blog.

While in the previous two posts we focused on the Mitsubishi i-Miev’s driving logic and the development of controlling algorithms for the Mitsubishi i-Miev test car, this time we will be introducing how the Iseauto bus was made in cooperation of TalTech’s School of Information Technologies and School of Engineering.

In the first post we talked about the need to first dismantle the existing Mitsubishi i-Mievi body to build a bus. Then you need to construct the body for the bus and finally install it. In all this, you need to take into account that the upcoming bus must be able to accommodate the hardware needed so that it can be continuously upgraded during its development. It is also important to hide all the hardware from the eyes of the passenger. In conclusion, designing a practical, user-friendly and expressive bus is a challenge for engineers and designers alike.

Silberauto AS, who, as we already know, is the largest partner in the project given to TalTech, was in charge for the design and production of the car body. The following pictures show how the molds were made, how they look like and how the final body of the car was mounted on the undercarriage.

Two big metal boxes were placed in the front of the car to accommodate the necessary hardware. One box is a personal computer, and the other for controllers and other equipment needed.

The control of the bus is to a large extent similar to the test car, but still has a few significant differences. The most notable of these is the control of the brakes. In total there are three different ways to stop the bus.

If we used an electronic handbrake as an emergency brake on our test car, then on the bus the electronic handbrake doesn’t have enough stopping force and this solution is not durable enough. So, we decided to use the brakes of the Mitsubishi i-Miev for braking. For this, however, we needed to add an electronic brake pump to the car. With the brake pump, we can pump pressure onto the brakes, but this is not enough. In the first post, we mentioned that in addition to the brakes that the management of solenoids is also important. We made sure that we only need to manage two different solenoids so we can use the brakes. The logic for the solenoids is easy to understand – in order to create pressure the solenoids must be activated. If they are deactivated, the brakes are released. Because we needed to know how much force was applied to the brakes, we added a pressure sensor to the braking system. With the help of an analogue-to-digital converter, the pressure sensor allows us to see the force applied to the brakes.

In addition to the usual brakes, the bus has a different handbraking system. Namely, there are two handbrakes on the bus – one manual and one electronic. The electronic handbrake is controlled by a linear actuator (read more here) mounted on the bottom of the car. The manual hand brake lever is placed between the boxes for the hardware of the car. Both handbrakes are connected to the i-Miev’s original handbraking cable, by pulling the cable sufficiently the brakes activate. The steering wheel, gearbox, and acceleration management behave in exactly the same way as on the test car.

In essence, three different brakes, that we have on the bus were described. Why do we need so many of them? One of the most important aspects of Iseauto is safety and the more ways we can stop a car, the better. As already described, we can not manually turn the steering wheel or press down on the brakes. Therefore, getting the car to stop in the event of danger has to be done with electronic solutions in most cases. The only exception to this is the manual handbrake.

Besides the mechanical brake, there are three options on the bus for the application of brakes.

The first and so far most used emergency brake button is connected directly to the drive controller and when it is pressed, the drive controller applies the brakes with maximum force. The advantages of this button are that the braking force is high and the time needed to apply the brakes is short. Also, when the button is released, you can resume the ride immediately. The disadvantage, however, is that if the drive controller does not work for some reason, the bus cannot be stopped through this button.

The next option to stop the car is with the help of a large red emergency button, which we have placed on the front of the car. When this button is pressed, the engine is turned off in the vehicle and the electronic hand brake operates. The advantage is that, because the car’s engine is turned off, the car will continue to run in a free motion, so to say the car doesn’t receive additional power from the engine. However, the activation of the electronic handbrake lasts for several seconds, and therefore the braking distance is longer. However, the braking distance can be lessened, by also using the manual hand brake.

As the most recent solution, we still have a switch in the car, which disconnects the engine from the battery. For this solution, we can rest assured that the car switches to “idle” and the engine will not be used to move forward. However, the disadvantage is that no brake is directly applied and after pressing this button and it is necessary to delete the error code from the control unit afterwards. There has been no reason to use this button in testing yet.

Unlike the test car, there are some improvements to the bus. The first of these are LED strips that are applied to the front and rear of the bus. Each LED is accompanied by a controller that changes the colors of the lights according to the command given to it by the CAN (controller area network).

Secondly, it necessary to control the bus door, for that we also have an controller. This controller also reacts to commands given by the CAN.

Thirdly, 8 ultrasonic sensors have been added to the bus, unlike the test car. Four of these are located on the front and four of these are located on the back. To obtain and transmit information from the sensor, there are controllers one for the front and one for the back, that collect information from the sensors and output the distance from any object to the CAN in decimeters.

In all, the important components of the bus have been talked about. Next time, let’s look at what the bus looks like, how and what sensors were placed on the bus and what Iseauto is capable of and what is its future.

The Ministry of Education and Research and the Estonian Research Council are supporting the completion of the blog.

Last time, we briefly introduced the TalTech Iseauto project and the underlying Mitsubishi i-Mievit with its control logic. This time, we will introduce the self-made controllers installed in Iseauto, their needs and why one or the other controller is necessary for driving the Iseauto. The first tests are done with the testcar in the picture below, which has become our platform to help us test all our hardware and software. Since many experiments were needed during the first development phase, we have modified test car has to be used for both self-driving and manual mode, if necessary.

The use of microcontrollers in my car is inevitable because they help to generate the same signals for driving a car that is caused by a person when he rotates the car, presses gas or brake pedal or the changes gear. A microcontroller can have up to several hundreds of input/output pins, all of which can be programmed as needed. In the Iseauto project, we use the controllers produced by STMicroelectronics. The main reasons for using them are a wide range of different controllers, big community, a convenient development interface and, of course, our past experience with their use.

TalTech Iseauto has about ten different controllers already. Some of them are made by the car manufacturer, and we still use them, and some of them have been replaced by our own developed controllers. In this post, we focus on two main controllers – we call them Drive and Master. The drive controller’s task is to control equipment used for driving the car. The main task of the master controller is to communicate information between different controllers and the personal computer. The following figure shows Master Controller Version 1.2. On the left side you can see the power connectors and the 3 CAN connector and on the right side is the ethernet jack.

Master controller

The main task of the master controller, the transmission of information to the correct controller, depends on what kind of data a controller needs. Some of the data can be transmitted at a certain interval, the time-critical data needed to make quick decisions is transmitted immediately upon arrival.

The master must be able to communicate in both directions with other controllers, i-Miev’s electronic control unit (ECU) and with a personal computer (PC). The ECU communicates with other devices via the CAN bus, and the fastest way for PC to communicate with the controller via an ethernet. We decided that it would be most sensible to communicate with controllers via the CAN bus. Therefore, the first requiremet for  Master controller is that it has to support the ethernet interface, and it also has to have at least three CAN interfaces to communicate with the controllers and the ECU. We found that the appropriate controller is STM32F767.

 

The CAN bus used for transmitting information is capable of transmitting messages in essence with only one particular protocol. These messages have one 11-bit or 29-bit identifier and up to 8 bytes of message. We use 11-bit identifiers in the project. With ethernet it is possible to use different protocols, but the UDP protocol is suitable for us, which does not require an on-the-spot confirmation. On the one hand, there is no need to send an acknowledgment of receipt of a message because the messages are sent 100 times per second. If one message is not received, the delay in the car is about 10ms, which is not critical for a vehicle moving up to 20 km/h. On the other hand, each message comes with its own counter, which allows us to automatically detect the differences in the number of sent and received messages, and to identify possible problems in hardware and software.

Drive controller

Another and considerably more complicated controller is the Drive controller. As mentioned above, his task is to drive the car – in other words, drive the car’s accelerator, brakes, steering wheel and gear lever. When we got to the Mitsubishi i-Miev’s control logic last time, this time we look at how the controller can mimic this control logic.

Let’s start the accelerator. As mentioned in the previous post, we need to generate two analogue voltage signals, where the MAIN signal must be twice as high as the SUB signal. Such analogue voltage signals are generated by a digital-to-analogue converter (DAC). The drive controller uses the STM32F407 microcontroller which has two built-in DACs, which we use.

To rotate a steering wheel, we need to drive a power electric motor. The direction of current flow determines at which direction and the strength of current how fast the wheels are rotated. To do this, we use the H-bridge as an electronic switch to control the electric motor in both directions. We have installed an H-bridge, which allows us to control the current through pulse width modulation. Who wants to read more about the operation of the H bridge, then one place is, for example, here.

Transmission control is extremely simple, it is necessary to direct a high voltage level to the correct ECU input. If the microcontroller’s normal operating voltage is either 3.3V or 5V, then the gearbox’s high voltage level signal must be 12V. It makes things a bit more complicated. In order to switch the 12V with 3.3V, we use optocouplers on Iseautol, which will electrically disconnect the controllers from the control signal of the car. This helps to protect our controllers if the car system should cause interference or some components to spoil and cause Drive controller to burn down.

In the development of the braking system in our car test drive, we had to take into account that we would have left the car braks in the trench so that we could use them if necessary. Therefore, we decided to use the BMW electronic handbrake as a brake, which we placed in the car instead of a mechanical handbrake. Since the car’s electrical handbrake is usually driven through the CAN interface, but we did not have the specification for the handbrake, we removed the handbrake control electronics and control the electric powertrain on the handbrake directly. Since this handbrake engine must also be driven in both directions (on and off), the H-bridge has to be used. Since we did not use the built-in handbrake limit switches (to understand how much the handbrake was on or off), we added the flow control sensors for the motor control circuit. If the power consumption of the handbrake engine exceeded the reference value, then we stopped pulling or releasing the handbrake. We also need to know in which handbrake is. To do this, we use an encoder built into the handbrake that gives the impeller every half-turn of the handbrake engine. It allows you to read the engine speed, or find out the current position of the engine.

Now we had everything necessary for testing – we were able to rotate the steering wheel, keep it in proper speed and turn on the desired gear. At last we added a relay box to the car, which allows us to turn the self-driving mode into manual driving mode for development purposes.

The Ministry of Education and Research and the Estonian Research Council are supporting the completion of the blog.

Introduction

The 21st century has changed a lot about the digital world around us: desktop computers have been replaced by laptops, mobile phones with smartphones and there is no longer distant from the time where vehicles that do not have the steering wheel, pedals and other controls necessary to drive, are going to be on the streets. Such vehicles are called self-driving vehicles, and one such development is also being addressed at Tallinn University of Technology (TalTech). In the following posts, we will describe the completion of the TalTech Iseauto in  four parts. First of all, we will investigate what one electric vehicle hides and how we can take control of it.

The Idea of TalTech Iseauto project

In the spring of 2017, TalTech and Silberauto AS signed a cooperation agreement to develop the self-driving bus for TalTech’s 100th anniversary. This is a six-seater bus, based on the Mitsubishi i-Miev electric car rack and Silberauto AS-made restructure. The purpose of the bus is to travel independently on the TalTech campus. Also, there are no necessary equipments for driving, including the steering wheel, pedals, gear lever. The second major objective of the project is to increase competence in self-driven vehicles. Moreover, TalTech makes its own automobile a special feature that most of the development work is done by the students, which creates the opportunity to acquire teamwork experience and provide the practical skills necessary for work in the industry.

Mitsubishi i-Miev

The basis of Iseauto was an already existing electric vehicle. The Mitshubishi i-Miev was found suitable for the project. With the development of the TalTech vehicle as the “last mile solution”, the i-Mievi 47kW engine is sufficient for this purpose. The travel distance per loading is also slightly more than 100km, which allows a large part of the day to keep up the battery when traveling around a small area of land. Also, its small dimensions and turning radius allow easy handling on narrow roads inside the TalTech’s campus. However, as the goal was to make a bus from the car, it was necessary to start dismantling the hull. The result is an undercarriage in the picture, from which all the remains are removed.

Getting to know the Mitsubishi i-Miev’s driving logic

Getting to know Mitsubishi i-Miev’s driving logic was one of the first objectives of the project. Our goal was to find out what signals we need to transmit to the car’s electronic circuitry so that we can imitate some devices, such as the accelerator pedal or gear lever. To do this, we used a lot of so-called reverse engineering, where, in essence, we measured, for example, the gas signal coming out from the accelerator and its change in time when we pressed the gas pedal. We also had the opportunity to read messages from the car’s electronic control unit (ECU),  which messages where transmitted along the car CAN-bus. The CAN interface is a communication interface between the so-called “brains” of the car and other electronic units. These messages are short messages that are exchanged up to 100 times per second. One car can have several dozen ECUs, each with its own specific task. Each ECU issues messages with a different identifier, which helps to understand which ECU has transmitted the message. Understanding which message has specific identifier and what form it transmits,we had to use help from the Internet because we had no documentation, and it was not possible to get it from the car manufacturer. Unfortunately, there are also no explanations available on the Internet for all messages, and therefore we can only use a handful of information that we need to get to take over the management of the car.

The driving logic of Mitsubishi i-Mievi

As the car’s one of the most important parts in addition to the steering wheel is the gas and brakes pedals, we first established the acceleration pedal control logic. To do this, we pressed the accelerator connected to the i-Miev electronics circuit and looked at how the signals change. We found that the main signal of the accelerator pedal is twice as large as the sub signal.

It’s extremely easy to control a gearbox on an electric motor car. The ECU has one input for each gear, and if this input voltage level is high, then the corresponding gear is turned on, and if it is low, then the gear is off. On the CAN-bus, there is also a certain message from which you can get feedback on which gear is turned on. If there are several gears turned on or none of them, then this can also be known.

By investigating the behavior of the steering we decided to solve the rotation of the car by turning the steering motor direct. In order to make the car’s steering wheel easier to rotate, today, every car’s steering wheel has power steering. As the power steering is an electric motor, giving the flow in one direction it turns the wheels to one side and if the flow is turned around the wheels turn to other side. The information about steering wheel position can be read from CAN-bus there i-Mievs steering wheel sensor  tells you which position the steering wheel is in relation to the zero position with 0.5 degrees precision. Since it is in the position of the steering position, not the wheels, a small conversion must be made and we can use the steering wheel sensor to determine the position of the wheels.

Braking control on the car is probably the most difficult one. A lot of different sensors must work together, but let’s not get very technical. The essential parts of our brakes are the brake pump, the solenoids, and the pressure on their brakes in their work. How exactly these elements are used on Iseauto, already in the next posts.

I-Miev has a conventional mechanical wireline solution for handbrake, which you can find in the most of older and cheaper class cars between the two front seats on the right hand side of the driver. Due to the design of my car, the same solution must be left out and it is planned to replace the mechanical handbrake with one of the most commonly used electric handbrakes today.

With that, we have reviewed everything that we plan to start managing ourselves. How do we do that, the next time.

The Ministry of Education and Research and the Estonian Research Council are supporting the completion of the blog.