ToborTech: FTC Robotics

Overview

For 3 years in high school, I was an active participant in the FIRST Tech Challenge robotics competition as a member of team 8610 ToborTech. I joined the team near the end of my freshman year and became a formal member at the start of the 2021-2022 season during my sophomore year. I remained part of the team until my graduation after the 2023-2024 season.

This page highlights parts of my experience on the team. On average, I worked on team projects for 400+ hours a year, with the 495 hours during my first year being the highest among members. Naturally, the content below only spotlights some of my most interesting and important contributions and learnings.

About FIRST Tech Challenge

FIRST Tech Challenge (FTC) is a robotics competition under the FIRST (For Inspiration and Recognition of Science and Technology) organization. Thousands of teams made up of 7th-12th grade students compete in FTC from all over the world every year.

Each season introduces a new game, which typically involves the intake and deposit of various kinds of game elements on fields with different structures. Each match is played between 2 alliances, each made up of 2 teams. The first 30 seconds of each match are the autonomous period, during which robots drive on the field and complete game objectives with no human inputs, relying solely on pre-programmed instructions and sensor readings. The remaining 2 minutes are the driver-controlled period in which each robot is driven by 2 human drivers using gamepad controllers.

Each season kicks off in September and officially concludes in April at the world championship. During the seasons I participated in, my team’s first 4 competitions of each season were league meets, followed by a league tournament, the state/regional championship, and finally world championship.

I competed in 3 FTC seasons: FREIGHT FRENZY, POWERPLAY, and CENTERSTAGE. The official one-page game descriptions can be found here, here, and here.

About ToborTech

ToborTech was an FTC team consisting primarily of students from Lake Oswego High School that operated out of the high school’s robotics lab building. During my 3 seasons, each FTC team had a maximum capacity of 15 members. ToborTech received many applications each season, with recruited rookie members and graduating alumni balancing in numbers. As such, the team had around 15 student members each season. We were organized into roughly 3 subteams: software, hardware, and documentation/outreach, though some members also worked across subteams. In addition to student members, we also had 2 amazing coaches and several awesome mentors who guided us in both technical and non-technical aspects of the competition.

The FIRST Tech Challenge software development kit (SDK) was written in Java and built as an Android project. Therefore, we also wrote our robots’ onboard software in Java. Development and deployment largely happened in Android Studio, though I also frequently used Sublime Text when I didn’t need to connect to robots. The team used Git for version control and GitHub for hosting our source code.

In addition to building and programming robots, a central aspect of FIRST Tech Challenge is community and professional outreach. To that end, I had the opportunity to engage with many local and global community members through media, events, and demos, as well as learn from industry professionals through numerous company tours and speaker events.

ToborTech had a consistent success record, advancing to the world championship in Houston, Texas for 2 of the 3 seasons I was on the team, and the Oregon state/regional championship for all 3.

Technical Contributions

I was a member of ToborTech’s software subteam, first as a programmer, then as a co-lead. In particular, I led the development of our autonomous programs that allowed robots to maneuver the field and accomplish game tasks on their own. Of course, I also spent time writing code for the teleoperation of the robot during the driver-controlled period of matches, but the bulk of my time was spent on autonomous programs.

I wrote tens of thousands of lines of code during my time on ToborTech. Among those lines were pathing/scoring logic, odometry tuning, multithreading, codebase-wide refactoring, and a lot more. Below are some of the most noteworthy technical contributions where I led the development effort.

Computer Vision

My biggest and proudest technical accomplishment on ToborTech was drastically increasing the use of computer vision in our software from near zero. I experimented with a variety of algorithms to improve reliability of autonomous driving and scoring, as well as aid human drivers in controlling robots. I owned our entire computer vision stack for nearly my whole time on the team.

The sections below dive into the ways in which I used computer vision in ToborTech’s robotics software, followed by a section on my workflow in developing these programs.

Randomization Detection

Before each FTC game starts, officials roll a die. The result of this randomization determines where a game element is placed among possible positions on the field (i.e. left, right, middle) or the orientation of a game element (e.g. which side of a cone faces the robots). If a robot autonomously detects this randomization correctly and performs the corresponding task during the following 30 seconds, the team is awarded bonus points.

During FREIGHT FRENZY, my first season, the randomization task was a game element placed in one of three possible locations in front of the robot. Competition organizers released a TensorFlow Lite model for detecting the default game elements, which we initially used for randomization detection. However, the randomization bonus earned using custom game elements was (and remained in subsequent seasons) one of the single-item game tasks worth the most points, so its success rate became a high priority for the team. This meant the detection had to be consistently accurate and easily fine-tunable when lighting changed across venues.

After some exploring and experimenting, I found OpenCV to be a suitable alternative to TensorFlow. Since the target object in this detection task was always of one uniform color, the simplest approach was to find the largest object of the desired color in frame and compare its position with known boundaries. I created an image processing pipeline with the following steps:

1. Capture image frame from webcam
2. Convert image from RGB color space to HSV color space
3. Create a mask of the image using color thresholds to retain only pixels of desired hue, saturation, and brightness
4. Find contours in the image mask, which are connected regions of pixels that could be the target object
5. Filter out contours that are too small or in unexpected locations
6. Select the largest contour as the target object, then get the contour’s center coordinates
7. Classify randomization based on the horizontal coordinate (left, middle, right)

TensorFlow object detection and this OpenCV pipeline were used together during FREIGHT FRENZY to ensure accuracy. TensorFlow was phased out in POWERPLAY, the next season, since OpenCV proved to be a reliable replacement. POWERPLAY’s randomization task was to determine which of 3 sides of a cone was facing the robot. I designed the cone’s signal sleeve with 3 solid colors (green, yellow, and purple), one printed on each side. I made another color segmentation pipeline for this task. It found the largest contours of green, yellow, and purple in the image, and returned the color whose contour was largest in area.

In my final year’s game, CENTERSTAGE, the randomization detection was similar to the one in FREIGHT FRENZY. This time, I constructed a pipeline that cropped the image frame into 3 zones, then determined which zone had the largest contour.

Intake Targeting

On the robot’s onboard hardware, OpenCV color segmentation was much faster than TensorFlow models at processing images. So in addition to randomization detection, OpenCV also unlocked other latency-sensitive uses throughout the autonomous and driver-controlled periods. One of these was intake assistance, where webcam images allowed robots to autonomously intake game elements and sometimes even aid drivers in speeding up manual intake.

The autonomous period of the FREIGHT FRENZY game awarded points for the intake and delivery of game elements (yellow cubes and white balls). These objects were randomly scattered in a corner of the field with variations across matches, so pre-determined approach paths were often suboptimal, wasting precious time in games where every second mattered. To optimize the freight intake accuracy, the team installed an additional webcam, and I experimented with using its real-time feed to approach a specific freight instead of driving blindly hoping to find something in the robot’s path.

Through color segmentation with fine-tuned HSV thresholds, yellow and white objects were isolated as shown below. One specific contour was identified as the target freight using rules that optimized for intake efficiency. From there, getting the game element in the quickest way became a 2-step process: turning and driving.

1. The robot turned to orient itself directly facing the target freight. Since the webcam for freight detection was mounted in a fixed location on the robot, objects directly in the line of the robot’s intake mechanism were seen along a known axis in the webcam feed. I empirically derived a math equation to calculate how much rotation was needed based on the target contour’s horizontal coordinate. The first time this rotation calculation worked in testing was one of the coolest moments during my time on robotics, as I got to witness how camera readings tangibly informed the robot’s decision in an unknown random environment.
2. Similarly, I derived an equation that calculated the physical distance between the intake mechanism and the freight from the vertical coordinate of the freight’s contour center in the image. All that remained for the robot to do was to drive forward while spinning the intake to collect the freight.

In subsequent seasons, I continued implementing computer vision processes for autonomously targeting game elements to improve intake speed and accuracy. In addition to the autonomous period, these systems were also used during the driver-controlled period to help drivers intake game elements quickly when the robot was in distant corners.

Delivery Assistance

Similar to how computer vision was used to aid the intake of game elements, I also developed OpenCV pipelines to assist in the delivery of game elements.

Alignment

During the FREIGHT FRENZY game’s autonomous period, freights were delivered to a shipping hub made up of 3 plates stacked vertically on a pole with space between the plates.

One problem we encountered was that after each delivery, the weight of the freight or contact with the robot sometimes knocked the hub off balance, tipping it slightly to the side in unpredictable ways. As a result, freights delivered in later delivery cycles had a higher chance of landing outside the intended plate. To address this issue, I used the robot’s webcam to improve delivery aim.

Once again using color segmentation, the red/blue center pole of the shipping hub was isolated as a contour in the webcam image, as shown in the slider image above. The robot used the difference between what the contour’s horizontal coordinate was and what it should ideally be to calculate how much the robot needed to turn to be in the optimal delivery position.

Target Selection

During CENTERSTAGE, both teams of an alliance delivered pixels (the game element that season) to “columns” on the same tilted board. In scenarios where our alliance partner had already delivered their preload pixel, it was strategically optimal to deliver our preload pixel to a specific column next to theirs. This both set the base for a bonus-awarding mosaic pattern later and avoided knocking off or moving our partner’s pixel.

Around that time, my teammates and I were experimenting with HUSKYLENS by DFRobot, a camera module with onboard image processing, for computer vision tasks as a complement/alternative to traditional webcams. We ended up using a HUSKYLENS for this target selection task. We used the module’s built-in color detection and AprilTag detection to identify 1) the position of any yellow preload pixels on the board and 2) the position of AprilTag markers under the delivery columns. These positions were compared to determine whether a column was occupied, and if so, which column to deliver our pixel to for maximum scoring.

Localization

Another important use of computer vision in our team’s software was localization, the process of determining where something was on the game field.

Self-Localization

Self-localization is the process through which a robot determines its location on a map. Our robots used to rely on odometry and inertial measurement units (IMUs) for this task, where unpowered freely-spinning dead wheels tracked displacement from the beginning of a match to produce the robot’s latest position and heading (which was further corrected by the IMU system). While this approach was accurate enough most of the time, any error in the tracking accumulated and compounded over time. The CENTERSTAGE season saw the introduction of AprilTags as part of the game field. These QR-code-resembling marker images were located in fixed places on the field. We took advantage of those tags to re-localize the robot in the left-right axis during autonomous deliveries, improving the accuracy of our localization based on the known positions of AprilTags. If the target AprilTag was not visible to the camera, the robot could also fall back to localizing off any of its neighboring tags instead.

Duck Localization

One of the coolest things I got to work on at ToborTech was the position estimation of fallen rubber ducks during FREIGHT FRENZY. One of the autonomous tasks that season was spinning a carousel to drop a rubber duck into the field. In addition, delivering the duck to the shipping hub also scored points. So in situations where our alliance partner was tasked with intaking and delivering cubes and balls from the corner warehouse to the shipping hub, we wanted to maximize scoring by delivering the carousel’s fallen duck onto a shipping hub.

The problem was that rubber ducks fell, bounced and landed in unpredictable ways. Sometimes, they ended up in particular areas near the field corner where our robot’s intake mechanism could not directly collect the duck. For this scenario, a “duck arm” was installed on our robot for sweeping ducks out of those dead zones.

In order to detect whether this sweeping routine was needed, I programmed a computer vision process for determining where a duck was on the field. The duck’s global coordinates were calculated by combining the robot’s own pose (coordinates and heading) and the duck’s position relative to the robot (angle and distance) as derived from the camera’s feed. These global coordinates were then geometrically compared against fixed dead zone regions, enabling the robot to decide whether it could directly collect the duck, or needed to perform a sweep first.

This procedure, where a camera identified not only an object’s relative position, but also its global coordinates on a “world map”, was eye-opening to me.

Development Workflow

Most of the computer vision tasks I worked on used the OpenCV library. For those, I spent much of my time tuning image processing pipelines and parameters using test scripts written in Python. I ran these scripts on my laptop with either the live feed from an attached webcam or previously-captured screenshots of the feed from a robot’s onboard webcam. When everything was working as intended in the testing environment, I then ported the Python code into Java and integrated the pipelines into the robot’s onboard software.

The computer vision parameters I modified the most by far were the HSV color thresholds for color segmentation. I wrote a dedicated Python script for finding the optimal HSV thresholds of colors. The script launched 3 image windows: original image with contour lines, a black frame with contour lines, and a masked image with a dot at the center of the largest contour. A fourth window contained a control panel with 7 sliders: 6 controlling lower and upper bounds of the H, S, and V parameters and 1 controlling the blur applied to the image. This script enabled swift and systematic tuning of parameters even when the robot was elsewhere for repairs, judging, or gameplay.

As computer vision became more and more prevalent in the team’s codebase, I created and began maintaining a utilities file for common image processing methods. By the end of my time on ToborTech, the file had accumulated over two dozen HSV threshold boundaries for colors that required detection over the seasons. Additionally, the file contained a collection of helper methods I extracted from repeated usage across our codebase, ranging from initializing webcam hardware with a processing pipeline to numerous ways of grid cropping.

/**
 * Get an OpenCV-acceptable mat from a Vuforia frame
 * @deprecated CENTERSTAGE pre-season (2023/8)
 * @param vuforia VuforiaLocalizer instance
 * @return mat
 */
@Deprecated Mat getMatFromFrame(VuforiaLocalizer vuforia)

/**
 * Create and initialize an OpenCvWebcam
 * @param hardwareMap hardware map
 * @param webcamName  name of webcam in hardware configuration
 * @param pipeline    OpenCvPipeline to attach to the webcam instance
 * @param width       width of the camera resolution
 * @param height      height of the camera resolution
 * @param orientation orientation of the physical camera
 * @return OpenCvCamera instance
 */
OpenCvCamera createInitializeWebcam(
  HardwareMap hardwareMap, String webcamName, OpenCvPipeline pipeline,
  int width, int height, OpenCvCameraRotation orientation
)

/**
 * Get center point of contour
 * @param contours list of contours
 * @param mIndex   index of specific contour in contours
 * @return (x, y) of center point of specified contour
 */
double[] getCenterOfContour(List<MatOfPoint> contours, int mIndex)

/**
 * Get area of largest contour of a certain color
 * @param mat  input image
 * @param low  lower bound of color
 * @param high higher bound of color
 * @return area of largest contour of specified color in image
 */
double getLargestAreaOfColor(Mat mat, Scalar low, Scalar high)

/**
 * Find list of contours of a certain color
 * @param mat  image
 * @param low  lower bound of color
 * @param high higher bound of color
 * @return list of contours
 */
List<MatOfPoint> findContours(Mat mat, Scalar low, Scalar high)

/**
 * Crop a frame
 * @param mat  input frame
 * @param rect rect representing region of interest
 * @return mat
 */
Mat crop(Mat mat, Rect rect)

/**
 * Crop a frame into 2 zones (horizontally)
 * @param mat original frame
 * @return array of 2 Mat objects (left, right)
 */
Mat[] cropInto2Zones(Mat mat)

/**
 * Crop a frame into 3 zones (horizontally)
 * @param mat original frame
 * @return array of 3 Mat objects (left, middle, right)
 */
Mat[] cropInto3Zones(Mat mat)

Autonomous Menu

Strategy is of paramount importance in FTC games. This was especially true during the POWERPLAY season, where the main task of the game was to place cones on poles or on the ground. A significant number of bonus points were awarded to an alliance for creating a circuit of connected cones across the field, and having the topmost cone on a pole/ground junction also resulted in bonus points. As a result, each cone could play a crucial role in the final score of a game. Even during the autonomous period, where each cone was delivered had to be intentional to both maximize total points and avoid collision between alliance partners. Specifically, the junctions most suitable for delivery were different when we wanted to optimize for early-game spread across junctions vs when we wanted the highest amount of raw points. While some alliances ended up having conflicting autonomous routes leading to one team sitting idle for the first 30 seconds, we wanted to be a flexible alliance partner and always have a compatible program to run no matter the route of our partner’s autonomous program.

Traditionally, a team had 4 sets of autonomous programs (OpModes), one for each combination of alliance (red/blue) and starting position on the field. At the start of a match, the drive team would select the desired OpMode, initialize it, and hit run when the buzzer sounded. To modify the driving path slightly or change where game elements were delivered meant writing a completely new program for each of the 4 possible starting locations. The POWERPLAY game field had over two dozen possible delivery locations (though each alliance only got about half the field during autonomous), so creating a new OpMode for each driving route we might want to run was simply not feasible.

In a prior season, our autonomous program got a new feature: autonomous menu. It was a clever OpMode that allowed the drive team to configure all the autonomous parameters: alliance color, starting position, parking position, and toggles for some telemetry logging. During competitions, our drivers were able to set up each autonomous run to behave exactly as needed without the overhead of new OpModes.

For POWERPLAY, I extended the use of the autonomous menu beyond just configuring parking position. Each pole/ground junction had a fixed and known location on the field. While the spot that the robot needed to be at to deliver to each junction varied by where the robot started, those delivery coordinates were also fixed and known. Furthermore, as we analyzed scoring strategies, the most valuable poles for autonomous delivery all sat on two sides of the path connecting each alliance’s two cone stacks. As such, once the robot was on that line, it only needed to drive along it (with some minor side-to-side adjustments).

This diagram shows the layout of the POWERPLAY game field with each junction labeled with its height. Drivers were positioned on the left and right sides in this image. Pole label indices were ordered from their perspectives.

To allow our drive team to select the poles for autonomous delivery, I created a HashMap that labeled field coordinates not unlike how geospatial mapping services geocode addresses into coordinates. I discussed with the team and we decided to use simple informative labels like H1, H2, M1, L1, etc (the prefix was the height of the pole: high, medium, low; the suffix was sequenced based on the drivers’ POV). For each of the 4 starting positions, the HashMap was populated with the labels of the 5 poles in its quadrant and information on each delivery position: horizontal and vertical coordinates, heading, arm angle, and arm height. Those roughly 100 parameters were tested and refined over the course of the season.

Lastly, I added controls for route configuration into the autonomous menu. When the drivers set the robot’s starting position, the menu automatically populated with a corresponding preset path. Then, using gamepad buttons, the drive team could configure the number of deliveries and rotate each delivery through the possible pole choices for the starting position. Once the targets were set, the robot would query the HashMap to find where it should go for each delivery and drive the specified route.

Just like that, our team became, to the best of my knowledge, the first to achieve on-the-fly autonomous route planning with selectable scoring targets. This gave us a great deal of flexibility in our scoring strategies and compatibility with alliance partners.

I adapted this system for the CENTERSTAGE game the following year. Though customizable delivery positions didn’t make sense for that game, the menu gave our drivers the option to configure delay time, parking, and cycle count so we could work well with other robots.

Internal Tooling

In addition to developing the software behind robots, I also spent much time writing programs for miscellaneous tasks that didn’t run on the robots themselves.

Discord Bot

One notable such utility program was Tobot, a bot I created for the team’s Discord server.

Its 4 suites of features were team lookups, competition resources, team-specific shortcuts, and scheduled tasks.

I wrote about Tobot in more detail here.

Scoring Apps

Another set of internal tools I built for the team was scoring apps. During qualifying tournaments and state championships, we were often ranked high enough at the end of qualification matches to be an alliance captain. This meant we got to invite partner(s) to join our alliance. To do so effectively, we needed to know about the other teams in the competition: strengths, weaknesses, consistency levels, autonomous routes, strategies, scoring habits, and much more. To find these insights, teams, including ours, sent designated team members to watch every robot during every qualifying match and track what they did in a process called scouting.

Traditionally, our team’s scouters recorded data by either completing Google Forms or filling out tables on paper. The results were then imported or manually entered into Google Sheets for analysis.

Though I was mainly involved in scouting only during my first season, my experience filling out scouting forms during my first few tournaments and entering data into Google Sheets in later events revealed many steps where the scouting process could be improved. For example, many input fields were numerical in nature: number of freights scored at a particular level of a shipping hub or number of cones delivered to low junctions during autonomous. Yet, the constraints of Google Forms and pieces of paper meant scouters had to either memorize dozens of numbers throughout the match or continuously erase-and-rewrite/delete-and-retype numbers into text input fields as games progressed. Furthermore, errors often slipped through during scouting since scouters had no reasonable way to sanity-check their data before the next match began, and many results had to be double-checked during data entry.

With those problems in mind, I decided to build a more customizable in-house scouting system for the CENTERSTAGE season. A few months before, I had already developed a non-scouting scoring app for the POWERPLAY season, which laid a solid foundation for the new scouting app. Since these apps were for internal use, I made them with Streamlit, a Python framework for making data-oriented web apps, because of its simplicity and maintainability.

Much of the interface remained similar to the Google Form version, but several changes were made that greatly improved efficiency:

• All inputs initialized to either 0 or the most common value. This saved scouters the time and mental bandwidth of typing 0 in many input fields for every robot every match during every tournament.
• Numerical inputs were actually numerical instead of text inputs. Scouters no longer had to click on an input field, delete the existing number, and type in a new number. Instead, they just hit the + button.
• Scores were calculated live as data was entered. This helped prevent mistakes since scouters could easily notice if they mistyped a number leading to an unrealistic score. This also meant the app could be used as a score calculator during strategy discussions and practice matches.

Since our analysis formulas were already in place in Google Sheets, we wanted to keep that as it was. Additionally, for development simplicity mid-season, I decided to not mess around with the Google Sheets API. Instead, the app generated a pre-filled Google Form link using the data in the input fields. The scouters only needed to click on the link and click submit, and the data would end up in the form’s associated spreadsheet.

Despite requiring an extra click, the new app quickly became the default for our scouters. This screenshot shows the user interface for entering scouting data.

Data Tools

Another collection of internal-facing software I built while on ToborTech was data apps. To determine the most effective strategies both in developing robots’ software and hardware, as well as in planning scoring approaches in matches, we needed data.

I wrote Python scripts to scrape and pull scoring data off ftcstats.org and The Orange Alliance. These data were then shown in Streamlit data web apps and standalone data visualizations.

One way in which my data tools were used was measuring our scoring capabilities and comparing them against those of other teams around the world. That way, we could analyze our strengths and weaknesses and best decide what game tasks to focus on optimizing ahead of the world championship. The graphs below show the distributions of various scoring metrics from thousands of teams who participated in the FREIGHT FRENZY season. The green lines mark ToborTech’s performance for comparison.

Miscellaneous Engineering

The technical contributions highlighted above are some of the more interesting software work I did on ToborTech, but very far from the only programming I did. Much of my time was spent working on a wide range of miscellaneous tasks.

I spent an enormous amount of time on the programming and tuning of autonomous routes: running an autonomous program, watching a video recording for irregularities and places to optimize, searching through run logs for what happened, tweaking some code, resetting the robot and the field, then doing it all over again. This process often repeated a dozen times in each meeting. While not as exciting as writing a demo program for the robot to follow a cube in my hand, this type of repetitive fine-tuning was what built much of our team’s success on the field.

One of our team’s most important design philosophies was the use of a wide variety of onboard sensors to bolster autonomous capabilities and to assist drivers during the driver-controlled period, which was also among the things recognized by the Control Award. Throughout my 3 seasons, our robots were equipped with camera modules, dead-wheel odometry encoders, inertial measurement units, range sensors, motor encoders, voltage sensors, proximity sensors, optical distance sensors, color sensors, physical limit switches, magnetic sensors, and infrared sensors. I did not work with all of those, of course, but I did gain working knowledge of a good number of them while using them in programs. In particular, besides camera modules, I spent a notable amount of time calibrating our odometry module, arguably the most important sensor during autonomous.

Some of my work never made it onto the competition field. One such example was my off-season experimentation with the motion planning library Road Runner and the observability platform FTC Dashboard. On the computer vision side, while only color detection OpenCV pipelines ran during matches, I also spent much time experimenting with circle detection.

It would be difficult to list everything else I contributed to ToborTech, from chassis motion primitives and hardware module configurations to a Streamlit dashboard for visualizing per-team/per-event scoring breakdowns. What I wrote about above are only the highlights of the code I wrote in 3 years.

Outreach Contributions

Robotics is the core of the competition, but FIRST is about more than robots. In addition to excelling in the building and programming of robots, ToborTech members also dedicated remarkable hours and effort to outreach. We engaged communities near and far through educational workshops, fundraising for worthy local and international causes, community events, volunteering to host competitions, social media, and more. Alongside my teammates, I was actively involved in these types of outreach events. In addition, I also drove a number of community outreach initiatives directly through media and knowledge sharing.

Media Production

I was the team’s unofficial photographer and filmmaker. I went to most competitions and events with a camera in hand to capture candid moments of the team.

During the 2022-2023 POWERPLAY season, I recorded enough footage of the team to create a 50-minute season vlog. I also created vlogs of competitions like the 2021-2022 state championship and world championship and several competitions during the 2023-2024 season. Though I made these mainly as keepsakes for the team, their reach went further, with one Australian viewer even commenting on the FREIGHT FRENZY world championship video that it hyped their team up ahead of their own worlds appearance the following year.

Besides videos of the team, I produced reveal videos of our robots too. Some of these videos became quite popular in the FTC community, with the first CENTERSTAGE robot reveal receiving over 10,000 views in its first 20 days of release. Here are the videos:

Additionally, I made a number of videos for miscellaneous purposes like a looping demo video for our pit station at competitions, videos submitted for awards, a marketing video for an international outreach initiative, demo videos played during judging presentations, a robot showcase featured in a community reveal night livestream, videos posted on team social media, etc.

Last but not least, I created a Java Basics course with 35 videos on various topics, linked here.

YouTube Channel Management

On top of making videos, I started overseeing the team’s YouTube channel during my first year on the team. Compared to the 7 years before, 3 years under my management saw our channel’s content receive 20 times the views, 23 times the watch time, 12 times the subscribers, and 51 times the impressions. Over the years, videos I made received comments from audience members all around the world ranging from school friends to teams from Australia and Brazil. Of course, this growth, along with the global brand that our team came to have as a result, did not come solely from my work. But I’m confident that the time I spent creating videos, redirecting community comments to the right teammates, marketing/releasing content, and more, made meaningful contributions to this progress.

Community Outreach

My media production and YouTube channel management accounted for a major portion of ToborTech’s broader global outreach, reaching people from over 20 countries. In addition, I was also a part of many of the more local community outreach events we ran. I helped showcase our robots during community events at schools around the area to inspire the next generation of engineers. We also had a continuous relationship with Ronald McDonald House Oregon, through which we routinely visited their facilities with demo robots and gave the children staying there a chance to drive them around and learn about robotics. ToborTech also hosted or co-hosted a number of events, including official competitions, international scrimmages, and regional season kickoffs, all of which I contributed to.

Finally, I was one of several ToborTech team members who worked as counselors at a robotics camp in the summer of my freshman year, not long after I joined the team. Organized by Lake Oswego Robotics, the umbrella non-profit that supported the community’s robotics teams, the camp aimed to teach elementary school students the basics of robotics. I was a camp counselor at River Grove Elementary School for 2 weeks, where I taught 2 dozen students alongside 2 other counselors. The core curriculum was centered around programming LEGO robots that the counselors built ahead of time, focused on the use of various kinds of sensors to navigate obstacles and paths. Once the students completed the key objectives we prepared, I led a few supplementary Python lessons in the later days of the camp.

Learnings

The 3 years I spent on ToborTech taught me a lot. Below are just some of the many lessons I brought with me from being on the team.

Technical Learnings

On the technical side, I gained much proficiency in Java and Python programming and various libraries in their ecosystems.

Our codebase was the largest I had worked with at the time. The combination of code specific to individual robots and shared utility code further added to the complexity of our software. I learned a lot about how to best organize programs. In fact, during the off-season after my first season, I led the refactoring of our code to make organization less confusing and more intentional.

My time on ToborTech was also my first time programming on a team. Merge conflicts were frequent, and I solidified my understanding of using Git for collaborative version control.

Likely the most important technical lessons I learned from ToborTech were on debugging. Bugs were common and hard to find in robotics programming. Some bugs occurred intermittently and only when certain unpredictable hardware components were under specific conditions. Because of this, determining the source of an undesired behavior was a significant and time-consuming activity during programming work sessions. I learned several useful techniques for spotting bugs.

• scrcpy was very helpful for viewing and reproducing what onboard webcams saw to identify issues with computer vision pipelines.
• Logging was also implemented and consistently used in our code, where informational logging statements were placed in notable checkpoints like the start of methods or after the program entered a branch. Matched with video recordings of the robots during runs, these logs gave important clues that helped pinpoint where the program was when an action happened or why the code went down a specific path.
• Telemetry, the process where a robot’s data is transmitted to the control station in real time, served a similar purpose and allowed us to monitor a robot and its sensors as a program ran.

Additionally, unexpected issues often came up in the midst of competitions, requiring emergency adjustments in between matches. Two more factors further complicated these software modifications: most competitions didn’t have practice fields, so changes often got deployed onto the robot without much testing if any at all; matches also happened in quick succession, adding time pressure to the mix.

ToborTech gave me the opportunity to work with a number of mathematical concepts related to robotics. Our robots used dead-wheel odometry, where unpowered dead wheels tracked a robot’s displacement in the horizontal and vertical directions. Combined with linear algebra, these 1D displacements allowed us to always have an estimate of the robot’s coordinates and heading. I also learned about PID control, a feedback control system used to tune our robots’ movements.

I learned much from the software I worked on. I thought: what if I shared what I learned with my teammates and they did the same? To that end, I spearheaded a software peer knowledge sharing initiative in the 2023 off-season. In the course of a few weeks, my teammates and I made a series of presentations on dozens of topics ranging from object-oriented programming basics to hardware mapping. I led a number of those lessons and taught my teammates about computer vision, version control, and more, but I also learned a lot of new things from my peers: hardware components like motors and servos and how they are controlled via code, our custom task queuing system, etc.

Learnings from Judging

In addition to technical concepts, I also learned much beyond robotics itself. Qualifying tournaments, state championships, and world championships all included a judging aspect. During these competitions, judges, typically industry professionals and/or FIRST alumni, learned about the work of each team in order to assign awards. This happened in 2 stages:

1. Judging presentations happened before robot matches. These were 10-15 minutes long in front of a panel of 2 or 3 judges, the first part being a rehearsed presentation and the second part being an interview-style Q&A.
2. Pit judging happened at the same time as robot matches. Judges, grouped by their expertise, approached teams in pairs or trios, and asked more in-depth questions about specific aspects of each team’s work.

These judging processes taught me a number of useful technical communication skills. I learned to document the progress of my work over the course of a season. I learned to compress an incredible amount of technical work done in 8 months into a script that was a few minutes long. I learned how to not just describe a feature but also concisely sell its usefulness and necessity in an overarching system. I learned to answer specific questions we did not prepare an answer for. Since pit judging happened without notice and without regard for who was at the pit, I learned to explain work I did not do while waiting for teammates with more relevant knowledge to return.

Learnings from Setbacks

By all measures, ToborTech was immensely successful in its last 3 seasons. I learned much through that success. Not everything worked all the time though, and I also learned much from setbacks. Some of these stories, like the 3 below, and the insights I gained from them, have stayed with me throughout the years since.

FREIGHT FRENZY State Championship Finals Match 3 Crash

During finals match 3 of our FREIGHT FRENZY state championship, the very last match of the event, and the one that could decide the fate of the team’s advancement to the world championship, the robot crashed and disconnected less than halfway through the autonomous period.

Ultimately, we determined the likely root cause of what happened to be instability in the electrical system, not directly related to the autonomous software itself. A mechanical component drew too much power from the battery, triggering a reset in the control system while the drivetrain motors were propelling the robot towards the wall, which resulted in the uncontrolled driving behavior observed.

I took away 2 main things from this event:

1. No matter how much preparation is put into the development of a piece of software, unexpected circumstances can still arise when it’s run in the real world. It is important to adapt to reality as the scenario changes, as our drive team did during the rest of that match to secure our eventual victory despite the rough start.
2. When something breaks, it’s important to get to the bottom of the problem. We spent much time after this match analyzing the root cause of the issue so we could avoid it in the future. In the subsequent seasons, we implemented changes in the robot software to reduce battery overdraw, such as chassis power scaling.

FREIGHT FRENZY World Championship Computer Vision Judging Demo

The computer vision work I did was one of the team’s software highlights during the FREIGHT FRENZY season. As such, our judging presentation during the world championship included a short demo of the intake targeting system.

The idea was that after I briefly explained how computer vision was used in our robot’s software, we’d trigger the robot to execute a predefined combination of actions: detect the freights in front of the robot’s intake through the webcam, determine which one to collect, turn to face it, then drive forward to intake it.

This process was rehearsed over and over again during judging practices with much success. The freights’ white and yellow were easily distinguishable from the gray competition field, but we also considered the possibility that the judging venue in Houston would have flooring of a different color. Our solution was bringing a couple of competition field tiles with us to judging, so that the robot demo would happen on the familiar gray tiles.

When judging came, everything ran smoothly until my demo. The combo was triggered, and the robot drove straight ahead and off the field tiles, completely ignoring the freights in its path. After we reset the robot and freights, the second attempt went the same way as the first.

Though the robot performed well in the following days’ matches and we ultimately placed as a finalist of the Control Award despite the demo mishap, the failure of what was supposed to be the highlight of our judging presentation still left me shaken. Looking into the code after the judging, I found the issue was something we could’ve easily avoided: after numerous iterations on the freight selection system by me and other members, the core logic became more of a high-level abstraction than a notable detail I remembered vividly. I assumed that the robot would select the freight closest to the intake mechanism, which was true early on in the module’s development when I was the only one working on the system. However, with changes made later by a teammate, the freight selection logic was modified to prefer freights straight in front of the robot over nearby freights requiring more rotation. This was optimal on the competition field, where turning slowed down the robot’s cycle time more than driving farther in the same direction.

What happened during the judging demo was that the robot’s webcam saw a small part of the light-colored venue flooring past the gray field tiles. The flooring’s color was close enough to the freights’ that the robot considered it to be an additional freight. The latest freight selection formula caused the robot to prefer this “freight” since it required less rotation despite being farther away. That is why the robot drove off the field tiles during the demo.

My main lesson from this experience is that code can drift over time especially in a collaborative development environment. Someone may adjust multi-purpose code to optimize for one purpose but unintentionally degrade the behavior of another purpose, particularly in edge cases. Therefore, it is essential to keep the whole team in sync about changes made to the software, even if they may appear trivial at first glance.

CENTERSTAGE World Championship Bugs

During CENTERSTAGE’s world championship, my code caused 2 mishaps in our robot’s onboard software.

The first bug was created when I made some changes to the AprilTag detection logic during the competition. The change was meant to address the scenario where the target AprilTag was not found but a neighboring tag was detected, in which case the neighbor should be used for localization. Long story short, part of the change was substituting aprilTagsData[id] with aprilTagsData[foundId]. I replaced all references to aprilTagsData[id]. Except for 2 instances in a telemetry log line. As this branch of the computer vision logic was intended to handle a hard-to-reproduce edge case, the limited field testing we could do did not expose the failure mode in time. As a result, a NullPointerException was thrown at runtime on the log line, crashing the autonomous program.

The second bug was another mistake I made with the AprilTag detection logic. At one point, due to hardware constraints, the robot’s backup webcam was installed upside down. Accordingly, I programmed the computer vision logic to account for this flip. Later, when the camera was re-installed right-side up, I never adjusted the software to match until midway through matches at the world championship. This bug was also hard to reproduce during testing because the flipped webcam was a backup for another camera, and its detections only kicked in when the other failed.

Unfortunately, both of these edge-case bugs were surfaced in real matches. Fortunately, I learned valuable lessons from them: manual testing can’t catch everything every time. Only extreme care, consistent cross-functional communication, and automated testing, where applicable, can prevent careless mistakes from ending up in production code. In particular, these 2 bugs were both on secondary paths, making them even harder to test in nominal conditions.

Industry Learnings

My 3 years on ToborTech also gave me exposure to the professional world. I had the opportunity to hear from professionals from various industries and companies, who shared both insights into their own careers as well as innovation happening all over the world.

Gracious Professionalism

Last but not least, I also gained an appreciation for Gracious Professionalism and Coopertition, foundational principles of FIRST. Watching coaches, mentors, volunteers, my teammates, and members of other teams during events, competitions, etc instilled in me the importance of cooperating kindly while competing fiercely, as well as graciously uplifting all members of the community whenever possible.

Awards and Achievements

During my 3 seasons on ToborTech, the team won around 25 awards in various areas at all levels of competition. Below are the state/regional- and world-level awards that are most relevant to my contributions on the team.

In the following photo, I was holding the Control Award finalist plaque at the 2022 FREIGHT FRENZY World Championship, one of my proudest achievements.

2021-2022 FREIGHT FRENZY

Oregon State Championship: We won on the game field as the captain of the winning alliance. We also won the Control Award, which recognized innovative use of software for robot control; and 2nd place for the Inspire Award, the highest award in FTC.

World Championship (Houston): We were recognized for our software again as a finalist of the Control Award. We also ranked #14 in our division by robot scoring.

2022-2023 POWERPLAY

Oregon Regional Championship: We won the Control Award and ranked 2nd for robot scoring. Note: this event was conducted remotely, with each team playing asynchronously on a modified smaller field.

2023-2024 CENTERSTAGE

Oregon State Championship: We were the 1st team selected of the winning alliance and were awarded 2nd place for the Inspire Award. The team’s head coach, Wan-Shu, was recognized for her mentorship of the team and awarded the Compass Award, for which I directed and edited the video submission.

World Championship (Houston): We won 3rd place for the Control Award in our division.

Unofficial Rankings

In addition to official awards, ToborTech’s robotics accomplishments were also commendable in unofficial scoring rankings. At various times throughout the seasons, our scoring capabilities were ranked as high as top 10 in the world, with some of our match scores ranked within the top 5 globally. In Oregon, widely recognized as one of the most competitive FTC regions in the entire program, we were consistently known as one of the strongest teams. Our scoring abilities were nearly always top 3 in the state, often reaching the very top and setting many state records.

Robot Videos

Below are some videos of ToborTech robots in action. More clips can be found in videos linked in the Media Production section.

FREIGHT FRENZY

The following videos show our autonomous programs from the warehouse-side and carousel-side starting positions. My computer vision work for intake targeting can be seen, most noticeably for intaking the duck in the second video.

POWERPLAY

The first 2 videos show one of our autonomous routes and our performance during the driver-controlled period.

The third clip is the video we submitted for our application to an off-season competition, where we scored 242 points by ourselves. For reference, the season’s world record score for a 2-team alliance was 311 points.

CENTERSTAGE

The following videos are of 3 matches ToborTech played.

In the first, a semifinal match during our league tournament, our alliance scored 334 points. As the emcee’s commentary pointed out at the end, this was one of the highest scores in the world at the time.

The second and third videos are 2 matches from the event finals of the Oregon State Championship. Our victories in those matches qualified us for the season’s world championship.

By the Numbers