What is TeleDrive?
The COVID-19 pandemic left FIRST Tech Challenge competitions, scrimmages, and team driver practice in the dark. Because of local restrictions, many teams use online resources like Zoom to communicate and start preparing for the next season. However, there still isn’t a solution to being able to practice driving the robot as well as coordinating competitions and scrimmages – all remotely… until now! Welcome to TeleDrive.
TeleDrive is a simple, yet powerful tool that allows teams to control their robot from the comfort of their homes. Using this, competitions can be run remotely and teams can drive their robot without having to worry about personal health and safety. Our program uses simple commands to bring low-latency and high-performance control, just as if controlling your robot on the field.
How does TeleDrive work?
TeleDrive has two main parts: sending the movement commands and receiving the live view of the robot moving on the field. By utilizing the Internet’s great power, all of these happen with less than 25-millisecond latency. How do we do it? Great question.
Sending Gamepad Commands to the Robot:
The first and main part of TeleDrive is the transmission of gamepad commands from the sender (each of the drivers’ laptops) to the robot, using the Internet. Each part of this is written such that drivers can control the robot with minimal latency. TeleDrive is built upon UDP, an internet protocol that allows data to be sent by connecting to another IP address and port. This allows for sub-15 millisecond control. Unlike TCP, if data is not received, it wouldn’t wait to resend the data which results in extremely low latency. In this setup, there are three computers required: the two driver-controlled computers (on the left side) and the “middle-man” computer (on the right side). On each of the computers – the driver controller computers, and the “middle-man” computer – a script is running in the background to either listen or send data. On the driver controller computers, the script will send to the specific IP address of the “middle-man” computer. The “middle-man” computer, which is listening on a UDP socket, immediately relays the data to the robot controller (or Control Hub) as it is also connected to the robot controller’s IP address. To easily convert these gamepad controls, we have created a Java class that reads the data sent and sets the “gamepad1” and “gamepad2” variables accordingly. Since this code extends the “OpMode” class, you can just extend our class to get the full functionality of your TeleOp code with UDP without changing any of the preexisting OpMode code.
Receiving Extremely Low-Latency Live Stream of the Robot:
The second part of TeleDrive is being able to see how your robot is moving on the field without having to deal with the latency of the image being sent back. With the low latency transmission of commands from the driver controllers all the way to the robot, remotely, it has very little latency and we wanted the live camera feed to be the same. To do so, we use an app called Parsec which is able to send the live stream to multiple computers connected to the host (laptop on the left side). Parsec is an application that uses UDP, the same protocol as sending the commands, but has one key difference which allows ping times under 25 milliseconds: they didn’t build on top of any wrappers since they wanted full hardware control and the ability to manipulate every element of the stream to reduce latency as much as possible. Parsec acts as a screen sharing in our case where multiple driver controller computers become the clients for the host computer and receive the live stream from the host. From this, the drivers are able to watch their robot move while they click buttons on their gamepads all within 1/40 of a second!
TeleDrive in Real Life
Check out the TeleDrive Demo!
The Future of TeleDrive
Robot Controller App Proposal:
Our current system for sending the robot movement commands through the Internet is very simple. But that isn’t enough. We believe that if the Robot Controller phone or Control Hub were to be connected to the Internet, there should be the ability to send commands to init, start, and stop the OpModes remotely as well. Currently, one person has to be near the robot to start and stop the OpMode when ready and the use of the “middle-man” computer will then be redundant, simplifying the flow of commands from the driver to the robot. Essentially, if the Robot Controller app is changed so that it can receive commands via WiFi and through the Internet, it would simplify the whole flow while reducing latency even more. With these changes to the Robot Controller app, it would be super-simple for teams to setup TeleDrive for their TeleOp code and get ready for remote competitions, scrimmages, and even driver practice.
Proposal to FIRST:
Because of the pandemic, we did not have a concrete solution for holding competitions and we believe that TeleDrive is made for just that. Since TeleDrive only needs one computer per driver at their remote locations and one computer to receive and relay the commands to the robot, TeleDrive makes it very simple to host competitions without having the drivers attend in person. Not only that, sending the video stream back is very simple – all the computer needs is Parsec, the application used for live streaming with low-latency, and Internet connection. We feel that TeleDrive brings a solution to the table in solving the problem of hosting competitions remotely during these challenging times of COVID-19. In fact, the only people who need to be at the competition itself would be the Referees, Competition Coordinators, and Judges. All teams would simply have to drop off their robot beforehand and connect to the Robot Controller directly through WiFi, as mentioned in the previous section.