Introducing Bluepilot! A Ford specific fork for Comma3X / OpenPilot

[Ajzride]

The goal of bluepilot is to develop, test, and stage Ford specific enhancements, validating them before submission to the SunnyPilot team for inclusion in the parent project.

bluepilot is based on SunnyPilot, and is not intended to compete with or replace SunnyPilot, rather it is a place for early adopters to help vet out updates that can be incorporated into SunnyPilot in the future. All the existing features of SunnyPilot are included and in tact.

Since the controls for Ford on the Comma3x are significantly different from all of the other manufacturers, Comma and SunnyPilot are having trouble incorporating the code suggestions we are sending them. It takes quite a lot of effort to validate that a change installed for a Ford will not have negative effects on a Honda or Toyota. To help get some of the more beneficial improvements into the hands of the Ford community faster, the team is launching bluepilot, a dedicated fork of SunnyPilot for Ford vehicles.

More information on the bluepilot fork, features, and install instructions are available at the GitHub homepage:

https://github.com/BluePilotDev/bluepilot/

installation of bluepilot is simple. Using the menu on your Comma3x, uninstall whatever fork is currently installed, then after reboot during the setup process, use the following URL to install from:

https://bit.ly/bp-stable

Note: If you are unfamiliar with what a Comma3x is, please see this thread:
https://www.f150gen14.com/forum/threads/openpilot-on-the-f-150-bluecruise-alternative.19342/

Sponsored

 

[Ajzride]

In addition to all SunnyPilot features, the stable branch of bluepilot includes the following enhancements:

• [Improved Ford Longitudinal controls] - logic to adjust stock OpenPilot single acceleration signal into separate gas and brake signals for much smoother long control on Ford vehicles.
• [Anti-Windup in Turns] - Logic to reset the EPAS back to zero when a human turn is detected. This prevents the EPAS from winding up and fighting to keep turning after the car has straightened up. Makes experimental mode and MADS safer to use.
• [Anti Ping Pong Logic] - Applies a blend of desired curvature and predicted curvature when no curves have been detected for at least the next 3 seconds. This greatly reduces steering wheel wiggle and ping pong.
• [Less Aggressive Lane Changes] - utilizes a blend of desired curvature and predicted curvature to achieve smoother, less aggressive lane changes.

The following features are currently being developed on the staging branch

• BlueCruise UI on compatible vehicles.
• Tunable ping-pong dampening using predicted curvature
• Adjustable high curvature reduction to cutting corners/inside of curves

 

[Ajzride]

What is"the Model", why does it change so often, and why can't we fix problem x/y/z on Fords?

It is helpful to understand some core aspects of the OpenPilot software package to determine which things might could be improved by vehicle specific controls, and which ones cannot.

There are 3 main components to deal with:

• The Model.
  • The model is a component of the software that analyzes the available data of the environment and the vehicle data (video, canbus data, accelerometers, etc) and determines what path the vehicle should take. The model is a core component of all OpenPilot branches (openpilot, SunnyPilot, dragonpilot, frogpilot, bluepilot and on and on). The model is produced by Comma.ai (the same company that makes the Comma3x hardware) and usually is denoted by a two-word nick name which is derived as some play on words for the feature that Comma was trying to focus on when they produced the model (Duck Amigo, FarmVille, Blue Diamond, etc). The model is not a typical piece of software you would read in a code editor like Java, or python, or Visual Basic, rather it is in a machine learning language powered by AI. Machine Learning and AI outputs are heavily influenced by the data the they are trained on. Each time Comma produces a new model, they are using either new data (gathered from all of the Commas driving around in our vehicles), or they have organized the data and put an emphasis on something different they want the AI to learn from. Typically, if the undesired behavior we are seeing is related to "what the vehicle should do" then it is model related and much more difficult for vehicle specific code to adjust. The most common example that comes up is not completely stopping at stop signs. The model is what sets the target speed, and if approaching a stop sign it decides to only slow down to 14 mph, then there is nothing a coder can do in the Ford logic to come to a complete stop. The Ford logic isn't analyzing the environment and seeing stop signs, that is "the model". Most of the time when the issue is the model, our only course of action is to wait for the model to get better.
• Control Code
  • The control code is where 95% of vehicle custom logic is implemented. This is the code that takes "what the vehicle should do" from the model and translates it into "how the vehicle should do it". Control code takes an "acceleration" and "target speed" from the model, and determines how much gas or brake to apply. Sometimes we can utilize control code to offset some model deficiencies. A current example is that the model frequently calls for too aggressive path on lane changes, resulting in jerky moves. Since one of the outputs of the model is "lane change state", the coders are able to use the control code to only apply a percentage of the models requested lateral movement into the vehicles steering commands because it is known there is a lane change active and it will be too aggressive.
• Panda
  • Panda is the safety portion of the software package. It continuously monitors all of the other programs within the software stack to make sure they are executing properly and not hung up, and then also monitors all signals that the control code asks for before it will allow them to be pass on to the canbus to ensure that the signals being sent will not cause a movement that is considered unsafe by having too high of change in acceleration, either on the longitudinal or lateral axis.

 

[Ajzride]

How do Ford Lateral Controls Work and Why Are They Such a Challenge for OpenPilot?

The previous post discussed the three main aspects of OpenPilot which were "The Model" , "Control Code (where the lateral planner lives)" and "Panda" along the role they play in OpenPilot. It will now be useful to understand how the Ford system works from an architecture standpoint, and how OpenPilot interfaces to it.

Much like Comma the Ford system has three components, where there is a model, control code (lateral planner), and safety code. For the Ford system "The Model" is an output from the IPMA (image processing module A) to the lateral planner which is inside the PSCM (power steering control module), which is built into the steering rack and pinion unit. For most other manufacturers (Toyota, Honda, etc) the lateral planner lives in the IPMA, along with the model and the output is a direct signal to control the steering wheel. Since Ford's lateral planner is in the power steering rack itself, we cannot bypass it and control the steering wheel directly. This means that when OpenPilot is interfacing with a Ford vehicle, it is not providing a signal to control the steering wheel like every other manufacturer the system supports, it is translating the output of the comma model into a data structure that mimics what the output of the IPMA from Ford would be, so that the lateral planner in the PSCM will execute the movements that we desire.

To further complicate things, there is not a single signal from the IPMA to the PSCM, rather there are 7 signals that describe the path the vehicle needs to take, along with some toggles about how it should go about attempting to follow that path.

• Curvature - A measurement of the radius of the road in front of the vehicle
• Curvature Rate - A measurement of how quickly the radius of the road in front of the vehicle is changing
• Path Offset - How far from the center of the vehicle is away from the center of the ideal path
• Path Angle - The ideal angle for the vehicle to move along to reach a path offset of zero
  • Edit: Lately I have come to believe that path_angle is actually the steering wheel angle to correct path offset and not the angle of the vehicle relative to centerline.
• Ramp Type - How quickly should the steering rack adjust to changes in control signals
• Precision Type - How strictly should the steering rack attempt to follow the described path
• Mode - Off, limited, extended.

By using the extensive data logging capabilities of the Comma3x, we have been able to monitor all 7 of these signals on BlueCruise and Lane Centering drives, and attempt to reverse engineer how Ford calculates each one. The first three are relatively easy to reverse engineer and calculate based on the names and a basic understanding of geometry. The last four however present a much bigger challenge. We do have a pretty good idea of when Ford toggles precision type and ramp type based on the planned path, and mode seems to be a variable that is designed for future use because it has no discernible effect on output. However the path angle variable is calculated using a proprietary model predictive controller algorithm that we cannot duplicate. In accordance with Murphy's Law, path angle is the most influential control signal at our disposal (and we don't know how to calculate it).

We were able to write some code that would read the signals Ford's model calculated out of the IPMA (because it does this 100% of the time, even when lane centering or BlueCruise is turned off) and write those out using the Comma3x so that we could force BlueCruise to be engaged 100% of the time, even on surface streets, neighborhoods, and really curvy roads. What we found is that Ford does not appear to have designed the system for non-interstate use, because it controls very poorly in those situations. Extensive testing is underway to see if there is a way to manipulate the PSCM into behaving in a way Ford didn't design it to so that we can have 100% hands free control in all situations.

 

[Ajzride]

Desired Curvature versus Predicted Curvature and why it matters for Ford

Due to the unique nature of controlling Ford (see above), the bluepilot team has been forced to think outside the box to address ping pong on Fords. We will us this post to try and explain the issue and give some background on the solution.

We will first start with discussing what are predicted curvature and desired curvature. Predicated curvature describes the path that the road follows (assuming no lane changes). On a perfectly straight road (which rarely exists outside of drag strips), predicted curvature would be 0.0. Desired curvature describes the path the vehicle needs to follow to drive down the road. The key difference is that desired curvature accounts for error correction. Even perfectly straight roads are usually crowned to help with drainage, meaning that a curvature of 0.0 would cause the vehicle to drift off the road to the outside edge because there was no accounting for the road crown. In theory, this distinction is not a problem, but the application is where the issue arises.

There are very few vehicles designed for hands free driving. Ford (designed from the ground up for BlueCruise), Tesla's FSD, and Chevy SuperCruise are the major players. Some European cars are starting to go hands free as well but of these, only Ford is compatible with Comma (Supercruise Chevy's all have encrypted canbus). All of the other systems that Comma is designed around are designed for hands-on assistance by the ADAS. Since these systems are not designed for hands-free, the control is less than precise. The best analogy we have come up with, is that the ADAS in Hyundai, Kia, Toyota, etc is akin to an old muscle car with worn our steering components. If you have ever driven one, there is frequently slop in the steering wheel, where you can move the wheel 10-15 degrees and the front tires don't turn at all. The Ford system is like a brand new sports car having tight steering with no slop. This causes an issue for Ford as the Comma system is designed around these sloppy ADAS systems. If a human were to drive around in an old mustang with sloppy steering for a month, then hop in a 2024 mustang, the steering would feel twitchy. The result of this is that the commands coming out of the comma are exaggerated.

Below are two charts showing predicted curvature and desired curvature coming from the Comma. Predicted curvature is in blue, and desired curvature is in red. The first chart is in a curve, and the second chart is on a straight away:

In the first chart, which is for some curves, the desired and predicted curvature match really well. But in the second, you can see the desired curvature swinging positive and negative around the predicted curvature which stays very close to zero on the straightaway. This exaggerated swing of the desired curvature is what causes ping pong for Fords.

The only way this gets truly fixed 100% of the time is for Comma to update the models to output a smooth signal designed for hands-free vehicles. Until that happens, we are stuck with looking for a work around.

To address the issue, we first tried switching to predicted curvature. This did not work, because there was no error correction and the vehicle would just slowly drift to the outside of the lane and off the road due to the drainage slope.

Our current solution is to blend the two curvatures. In the current releases of bluepilot this is at a fixed 35%. Starting with release 1.1( which we hope to push out by the end of September) the ratio with be variable (for straight aways and curves), and the percentages will be tunable from the menu so that you can dial in the percentages that work best for your road conditions and truck suspension.

 

[Ajzride]

RESERVED FOR FUTURE TECHNICAL CONTENT

 

[Ajzride]

HOW TO ASK FOR HELP OR REPORT A BUG

1) Be sure to state which branch of bluepilot you are on (Stable, Staging-1.1 etc)
2) Share a picture of a typed-out summary of the settings you have on the "Ford" or "BluePilot" menu page. This is Ford in Stable and Staging-1.x and BluePilot in Staging-2.x
3) Share a route where the issue or undesired behavior is exhibited. Please tell us what time stamp to look at (along with the time zone). Be mindful if you post a route in the main forum, people can see locations, so if it includes your house, you might want to DM it to one of the BP developers or choose another route that doesn't include a sensitive location.
4) Include a detailed description of the issue.

 

[Ajzride]

Post #3 updated with an primer on the model and other portions of the Comma code.

https://www.f150gen14.com/forum/thr...cific-fork-for-the-comma3x.24241/#post-457705

 

[Svt54]

Thank you for all your hard work. I installed this today and used it on my way home from work. A roughly 22 mile drive (15 in rush hour on 2 and 3 lane expressway, 6 on a 2 lane highway and one on a coutry road).

Initial thoughts compared to SunnyPilot Dev3c:
1. My ping pong issue on straight stretches of expressway seem to be gone.
2. Braking when approaching slower traffic is smoother.
3. Lane changes are smoother and less abrupt.
4. Upping the cruise control by 5mph with no traffic in front of me on the expressway with SP caused a down shift and fairly aggressive acceleration. With BP it is just a smooth and steady climb without the downshift. Should be better for mileage.
5. Using SP my turn off the 2 lane highway which is basically a wide u turn resulted in the "wind up" EVERY time, causing me to have to fight the truck continuing to try to turn for 5-10 seconds after leaving the turn and straightening out. Today on BP was the first time I have made that turn and been able to release the steering wheel as soon as the truck was on the straight section.

Obviously I am only a few miles in and have a lot of testing to go but my initial impression is that this has huge improvements in several areas that bugged me before. Keep up the great work! I will be running bluepilot from now on.

 

[HammaMan]

Hell yeah, congrats to everyone working on this.

 

[Svt54]

@Ajzride I drive quite a bit. I put about 500 miles a week on my truck on average with a blend of all driving types, but not much inner city. If you need any tests, logs or videos to assist in development, please let me know.

 

[powerboatr]

Hell yeah, congrats to everyone working on this.

Yes they are doing a huge amount of R &D
It's mind boggling for this guy
Well done all

 

[GMPowerboost]

I tried bluepilot out and loaded it Friday, being new to the Comma device and all I haven't had the chance to test other models for myself and experience some of the less than stellar aspects of them, but I can say with bluepilot as it is now, I wouldn't give it up and it is absolutely life changing to driving. Thank you all that have contributed and worked hard on this, it truly shows as an amazing product and works great.

 

[Spiffy]

Ajzride. Your time and contribution to this project is amazing. I appreciate your commitment and all of your personal assistance. I understood the frustration you guys had with the coordination between SP and OP. With ford being so different compared to others I also understood the lack of excitement from OP. Going this new route will be beneficial for us Ford users.

Going foward. I have been blending ACC with SP for a long time. The hybrid combo has proven to be simply amazing. Am I right in assuming that you can now focus more on the experimental features strictly aimed to ford?

I haven't tried longitudinal in months. Is it worth a look now?

 

[Ajzride]

Am I right in assuming that you can now focus more on the experimental features strictly aimed to ford?

My current focus is on getting proper steering so we can take any curve on the road hands free.

I haven't tried longitudinal in months. Is it worth a look now?

I still prefer Ford's ACC, but BluePilot ACC is a huge improvement over stock OP. I use BluePilot ACC when I have a lot of red lights and want to use experimental mode.

Sponsored