(Platform 1.0.0) Bug in the most recent VOXL2 ESC Firmware (Random motors reverse directions each arm)

[Eric Katzfey]

@John-Nomikos-0 Is this on a custom drone? Seems like perhaps a bad PX4 parameter somewhere. Maybe you can wipe everything and start from scratch?

[John Nomikos 0]

@Eric-Katzfey Maybe there's something with my ESC firmware parameters that's causing issues?

I'm going to try to flash an esc config that is used on a modalai drone to see if it's any different

Also, is there an easy way to go to the previous ESC firmware version without manually plugging in? I want to go back to see if this is due to the firmware flash in a time-efficient manner. I'm pretty sure there isn't but I wanted to make absolute sure

Also it is on a custom drone, but we got our drone flying with the exact same parameters on the previous ESC version on 1.0.0. Really is seeming like an ESC firmware issue to me.

[Alex Kushleyev]

@John-Nomikos-0

Yes i can provide you with old firmware for m0117 (the original firmware you had before). You will need your old ESC params. All this can be done right from voxl2 without unplugging. I will follow up tomorrow morning.

[John Nomikos 0]

@Alex-Kushleyev said in (Platform 1.0.0) Bug in the most recent VOXL2 ESC Firmware (Random motors reverse directions each arm):

Yes i can provide you with old firmware for m0117 (the original firmware you had before). You will need your old ESC params. All this can be done right from voxl2 without unplugging. I will follow up tomorrow morning.

Sounds good. I don't think we ever changed the ESC params on the old firmware version. So they would be whatever the ESC came out of the factory with. I'm assuming there are default parameters available for the old firmware, right?

[Alex Kushleyev]

@John-Nomikos-0 , yes i will provide the default params that are loaded during production.

[Alex Kushleyev]

@John-Nomikos-0 ,

I have uploaded the older firmware that was used during production of M0117-1 ESC : https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/dev/voxl-esc-tools/firmware/old/modalai_esc_firmware_m0117_1_v0_36_f75f1fdb.bin

The default params that are loaded with that are here : https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/params-v1/voxl-esc-params/old/esc_params_modalai_4_in_1_revb.xml (please note that this is not the same as the latest esc_params_modalai_4_in_1_revb.xml in dev or master branch.

(you can put the old firmware file into the following location: /usr/share/modalai/voxl-esc-tools/firmware/

Note that if you want to perform the ESC downgrade right on VOXL 2, you need to get both of these files on your VOXL2 and run the following commands:

cd /usr/share/modalai/voxl-esc-tools/

#disable PX4
voxl-esc enable-bridge

#upload firmware to each ESC ID (you will hear a sad beep after each update, which means the params are invalid - this is ok
python3 voxl-esc-upload-firmware.py --firmware-file firmware/modalai_esc_firmware_m0117_1_v0_36_f75f1fdb.bin --id 0
python3 voxl-esc-upload-firmware.py --firmware-file firmware/modalai_esc_firmware_m0117_1_v0_36_f75f1fdb.bin --id 1
python3 voxl-esc-upload-firmware.py --firmware-file firmware/modalai_esc_firmware_m0117_1_v0_36_f75f1fdb.bin --id 2
python3 voxl-esc-upload-firmware.py --firmware-file firmware/modalai_esc_firmware_m0117_1_v0_36_f75f1fdb.bin --id 3

#upload old params (one command to update params on all 4 ESCs)
python3 voxl-esc-upload-params.py --params-file <path_to_your_params_file>/esc_params_modalai_4_in_1_revb.xml

#detect the ESCs
python3 voxl-esc-scan.py

#re-enable PX4
voxl-esc disable-bridge

Your firmware has should now be updated to have the hash f75f1fdb, which is the original firmware that you had on the ESC and the old firmware esc_params_modalai_4_in_1_revb.xml should have been the firmware that you received installed during production.

Depending on the baud rate that you use in your latest ESC params, you may have to update it (the old baud rate was 250K but it could have been changed to 2Mbit, please double check).

Let me know if you have any issues in this process

Alex

[John Nomikos 0]

@Alex-Kushleyev Thank you so much for the help. I will try this next week. Have a great weekend!

[John Nomikos 0]

@Alex-Kushleyev said in (Platform 1.0.0) Bug in the most recent VOXL2 ESC Firmware (Random motors reverse directions each arm):

python3 voxl-esc-upload-params.py --params-file

So I successfully flashed the firmware on all 4 ESCs. But uploading params gave me this error

voxl2:/usr/share/modalai/voxl-esc-tools$ python3 voxl-esc-upload-params.py --params-file /esc_params_modalai_4_in_1_revb.xml 
Detected Python version : 3.6.9 (default, Mar 10 2023, 16:46:00) 
[GCC 8.4.0]
Found voxl-esc tools bin version: 1.2

INFO: Params file name : /esc_params_modalai_4_in_1_revb.xml 
INFO: Params file size : 6971 bytes


VOXL Platform: M0054
Detected VOXL2 M0054 or M0104!
Found previous connection information in .voxl_esc_cache ..
Prioritizing /dev/slpi-uart-2 @ 2000000
INFO: Scanning for ESC firmware: /dev/slpi-uart-2, baud: 2000000
Sending library name request: libslpi_uart_bridge_slpi.so
Received standard error event 2
Sending initialization request
INFO: Scanning for ESC firmware: /dev/slpi-uart-2, baud: 250000
INFO: Scanning for ESC firmware: /dev/slpi-uart-2, baud: 921600
INFO: Scanning for ESC firmware: /dev/slpi-uart-2, baud: 230400
INFO: Scanning for ESC firmware: /dev/slpi-uart-2, baud: 57600
INFO: ESC(s) detected on port: /dev/slpi-uart-2, baud rate: 57600
INFO: ESCs detected:
INFO: ---------------------
ID: 0, SW: 36, HW: 34: ModalAi 4-in-1 ESC (M0117-1)
ID: 1, SW: 36, HW: 34: ModalAi 4-in-1 ESC (M0117-1)
ID: 2, SW: 36, HW: 34: ModalAi 4-in-1 ESC (M0117-1)
ID: 3, SW: 36, HW: 34: ModalAi 4-in-1 ESC (M0117-1)
---------------------
ERROR: ESC firmware version 36 is incompatible with params supported by this utility
       Acceptable ESC firmware versions are:  [38]

I think voxl-esc-upload-params is checking for the version. I could modify it and try again, unless there is an earlier tool meant for this firmware

[Alex Kushleyev]

@John-Nomikos-0 ,

Yes, you are right, in order to load older params, you need earlier version of the voxl-esc tools. Please use this version of voxl-esc tools, you can check it out right onto your VOXL2 next to the latest voxl-esc tools (do not overwrite) : https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/tree/params-v1

Everything else should work the same with old tools.

Sorry about that, i forgot to mention it

Alex

[John Nomikos 0]

@Alex-Kushleyev Thank you. Successfully uploaded params

[John Nomikos 0]

Successfully was able to arm and fly with older esc firmware on default params. Thank you for all the help

[Alex Kushleyev]

@John-Nomikos-0

wow really? I am glad that it worked for you..

However, are you able to share the modified ESC params that did not work for you? the whole XML file. I think you should be able to attach it here or just paste it? As I mentioned before, there should not be a performance difference in the new ESC firmware and we have been testing it without issues, so I suspect the issue is in the new ESC params that you used..

I would like to figure out the issue, in case this comes up again.. thank you!

Alex

[John Nomikos 0]

@Alex-Kushleyev

Sure thing. Here's the entire XML file of params I was using on the new firmware that I could not take off with:

<?xml version="1.0" encoding="UTF-8"?>

<!-- Copyright (c) 2020 ModalAI Inc.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice,
   this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,
   this list of conditions and the following disclaimer in the documentation
   and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors
   may be used to endorse or promote products derived from this software
   without specific prior written permission.

4. The Software is used solely in conjunction with devices provided by
   ModalAI Inc.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

For a license to use on non-ModalAI hardware, please contact license@modalai.com -->

<EscParameters>

  <IdParams>
    <param name="id"              value="127"/>  <!-- 0-7 .. 127 means use hardware ID pins to read ID-->
    <param name="dir"             value="2"/>    <!-- 0=fwd, 1=rev, 2=fwd id-based, 3=rev id-based -->
  </IdParams>

  <UartParams>
    <param name="protocol_version" value="2"/>          <!-- reserved for future use -->
    <param name="input_mode"       value="0"/>          <!-- reserved for future use -->
    <param name="baud_rate"        value="2000000"/>     <!-- communication bit rate -->
    <param name="char_timeout_ns"  value="0"/>          <!-- not used -->
    <param name="cmd_timeout_ns"   value="100000000"/>  <!-- timeout for incoming commands before ESC will stop the motor -->
  </UartParams>

  <TuneParams>
    <param name="pwm_frequency"       value="48000"/>  <!-- switching freqency of PWM signal going to motors. 24Khz and 48Khz are only options for now -->
    <param name="vbat_nominal_mv"     value="14800"/>  <!-- used for sanity checking and limiting of voltage-dependent funcions -->
    <param name="num_cycles_per_rev"  value="6"/>      <!-- number of pole pairs in the motor. used for converting electrical frequency to mechanical rpm -->
    <param name="min_rpm"             value="2000"/>   <!-- minimum RPM that will be attempted, otherwise capped -->
    <param name="max_rpm"             value="35000"/>  <!-- maximum RPM that will be attempted, otherwise capped -->
    <param name="min_pwm"             value="70"/>     <!-- cap for minimum power to be ever applied. max is 999 -->
    <param name="max_pwm"             value="999"/>    <!-- cap for maximum power to be ever applied. max is 999 -->
    <param name="pwm_vs_rpm_curve_a0" value="366.028455487"/>  <!-- this is actually motor_voltage vs rpm curve.. using legacy naming -->
    <param name="pwm_vs_rpm_curve_a1" value="0.267931384448"/> <!-- Emax RS1306 3300KV with tri-blade 3x3x3 -->
    <param name="pwm_vs_rpm_curve_a2" value="5.77024053745e-06"/>
    <param name="kp"                  value="0"/>    <!-- RPM controller proportional gain -->
    <param name="ki"                  value="0"/>     <!-- RPM controller proportional gain -->
    <param name="max_kpe"             value="0"/>    <!-- maximum proportional erorr term (max is 999) -->
    <param name="max_kie"             value="0"/>    <!-- maximum integral error term (max is 999) -->
    <param name="max_rpm_delta"       value="2000"/>    <!-- cap for maximum rpm error used in RPM controller -->
    
    <param name="spinup_type"         value="1"/>      <!-- 0: traditional, 1: sinusoidal -->
    <param name="spinup_power"        value="130"/>     <!-- power used to during spin-up procedure -->
    <param name="latch_power"         value="70"/>     <!-- power used during latching stage of spin-up (out of 999)-->
    <param name="spinup_power_ramp"   value="8"/>      <!-- it will take ( 4096 / (spinup_power_ramp*10000) ) seconds to ramp sinusoidal start-up power from 0 to spinup_power -->
    <param name="spinup_rpm_target"   value="3300"/>   <!-- Desired RPM at the end of the sinusoidal spin-up procedure -->
    <param name="spinup_time_ms"      value="500"/>   <!-- Duration of the sinusoidal spin-up procedure -->
    <param name="spinup_bemf_comp"    value="0"/>      <!-- 0: disable, 1:enable back-emf compensation in sinusoidal spin-up procedure -->
    <param name="motor_kv"            value="2400"/>    <!-- kV value of the motor. used in back-emf compensation during spin-up -->

    <param name="min_num_cross_for_closed_loop" value="5"/>  <!-- exit latching mode of fixed power after this number of zero crossings -->
    <param name="protection_stall_check_rpm" value="700"/> <!-- if motor spins below this RPM, stall check will trigger and stop / restart the motor -->

    <param name="brake_to_stop"       value="0"/>             <!-- apply brake when stopping motor (or not) -->
    <param name="stall_timeout_ns"    value="20000000"/>      <!-- after spin-up, if no zero crossing is not detected for this amount of time, motor is considered stalled -->
    <param name="require_reset_if_stalled"      value="0"/>   <!-- require sending an array of zero commands to reset before next spin-up, if motor stalled -->

    <param name="tone_freqs"          value="[100, 115, 225, 250, 0,0,0,0, 0,0,0,0]"/> <!-- 200 is 2000Hz, max 255 -->
    <param name="tone_durations"      value="[30,  10,  10,  10,  0,0,0,0, 0,0,0,0]"/> <!-- duration of each tone in units of 10 milli-seconds. Poor naming!!! -->
    <param name="tone_powers"         value="[60,  60,  60,  60,  0,0,0,0, 0,0,0,0]"/> <!-- max is 255 -->

    <param name="dt_threshold_ns"       value="150000"/>      <!-- during start up, ignore inter-commutation times less than this val, probably noise -->
    <param name="max_dt_ns"             value="2500000"/>     <!-- min and max values for time between two commutations. these are used as caps -->
    <param name="min_dt_ns"             value="10000"/>
    <param name="dt_bootstrap_ns"       value="2000000"/>     <!-- filter bootstrap value for commutation dt during start up -->

    <param name="spinup_stall_dt_ns"    value="6000000"/>     <!-- during spin-up, if no zero crossing is not detected for this amount of time, motor is considered stalled -->
    <param name="spinup_stall_check_ns" value="30000000"/>    <!-- time after beginning of spinup to start checking for spinup stall -->

    <param name="alignment_time_ns"     value="0"/>           <!-- alignment time before spin-up -->
    <param name="timing_advance"        value="0"/>
    <param name="sense_advance"         value="0"/>

    <param name="demag_timing"          value="0"/>    <!-- unused -->

  </TuneParams>
</EscParameters>

[John Nomikos 0]

Apologies, this file is not up to date from when I left off. I changed spinup_bemf_comp to this
<param name="spinup_bemf_comp" value="1"/>

and power back to 80.

Everything else on this file should be the same. Here's updated:

<?xml version="1.0" encoding="UTF-8"?>

<!-- Copyright (c) 2020 ModalAI Inc.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice,
   this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,
   this list of conditions and the following disclaimer in the documentation
   and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors
   may be used to endorse or promote products derived from this software
   without specific prior written permission.

4. The Software is used solely in conjunction with devices provided by
   ModalAI Inc.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

For a license to use on non-ModalAI hardware, please contact license@modalai.com -->

<EscParameters>

  <IdParams>
    <param name="id"              value="127"/>  <!-- 0-7 .. 127 means use hardware ID pins to read ID-->
    <param name="dir"             value="2"/>    <!-- 0=fwd, 1=rev, 2=fwd id-based, 3=rev id-based -->
  </IdParams>

  <UartParams>
    <param name="protocol_version" value="2"/>          <!-- reserved for future use -->
    <param name="input_mode"       value="0"/>          <!-- reserved for future use -->
    <param name="baud_rate"        value="2000000"/>     <!-- communication bit rate -->
    <param name="char_timeout_ns"  value="0"/>          <!-- not used -->
    <param name="cmd_timeout_ns"   value="100000000"/>  <!-- timeout for incoming commands before ESC will stop the motor -->
  </UartParams>

  <TuneParams>
    <param name="pwm_frequency"       value="48000"/>  <!-- switching freqency of PWM signal going to motors. 24Khz and 48Khz are only options for now -->
    <param name="vbat_nominal_mv"     value="14800"/>  <!-- used for sanity checking and limiting of voltage-dependent funcions -->
    <param name="num_cycles_per_rev"  value="6"/>      <!-- number of pole pairs in the motor. used for converting electrical frequency to mechanical rpm -->
    <param name="min_rpm"             value="2000"/>   <!-- minimum RPM that will be attempted, otherwise capped -->
    <param name="max_rpm"             value="35000"/>  <!-- maximum RPM that will be attempted, otherwise capped -->
    <param name="min_pwm"             value="70"/>     <!-- cap for minimum power to be ever applied. max is 999 -->
    <param name="max_pwm"             value="999"/>    <!-- cap for maximum power to be ever applied. max is 999 -->
    <param name="pwm_vs_rpm_curve_a0" value="366.028455487"/>  <!-- this is actually motor_voltage vs rpm curve.. using legacy naming -->
    <param name="pwm_vs_rpm_curve_a1" value="0.267931384448"/> <!-- Emax RS1306 3300KV with tri-blade 3x3x3 -->
    <param name="pwm_vs_rpm_curve_a2" value="5.77024053745e-06"/>
    <param name="kp"                  value="0"/>    <!-- RPM controller proportional gain -->
    <param name="ki"                  value="0"/>     <!-- RPM controller proportional gain -->
    <param name="max_kpe"             value="0"/>    <!-- maximum proportional erorr term (max is 999) -->
    <param name="max_kie"             value="0"/>    <!-- maximum integral error term (max is 999) -->
    <param name="max_rpm_delta"       value="2000"/>    <!-- cap for maximum rpm error used in RPM controller -->
    
    <param name="spinup_type"         value="1"/>      <!-- 0: traditional, 1: sinusoidal -->
    <param name="spinup_power"        value="80"/>     <!-- power used to during spin-up procedure -->
    <param name="latch_power"         value="70"/>     <!-- power used during latching stage of spin-up (out of 999)-->
    <param name="spinup_power_ramp"   value="8"/>      <!-- it will take ( 4096 / (spinup_power_ramp*10000) ) seconds to ramp sinusoidal start-up power from 0 to spinup_power -->
    <param name="spinup_rpm_target"   value="3300"/>   <!-- Desired RPM at the end of the sinusoidal spin-up procedure -->
    <param name="spinup_time_ms"      value="500"/>   <!-- Duration of the sinusoidal spin-up procedure -->
    <param name="spinup_bemf_comp"    value="1"/>      <!-- 0: disable, 1:enable back-emf compensation in sinusoidal spin-up procedure -->
    <param name="motor_kv"            value="2400"/>    <!-- kV value of the motor. used in back-emf compensation during spin-up -->

    <param name="min_num_cross_for_closed_loop" value="5"/>  <!-- exit latching mode of fixed power after this number of zero crossings -->
    <param name="protection_stall_check_rpm" value="700"/> <!-- if motor spins below this RPM, stall check will trigger and stop / restart the motor -->

    <param name="brake_to_stop"       value="0"/>             <!-- apply brake when stopping motor (or not) -->
    <param name="stall_timeout_ns"    value="20000000"/>      <!-- after spin-up, if no zero crossing is not detected for this amount of time, motor is considered stalled -->
    <param name="require_reset_if_stalled"      value="0"/>   <!-- require sending an array of zero commands to reset before next spin-up, if motor stalled -->

    <param name="tone_freqs"          value="[100, 115, 225, 250, 0,0,0,0, 0,0,0,0]"/> <!-- 200 is 2000Hz, max 255 -->
    <param name="tone_durations"      value="[30,  10,  10,  10,  0,0,0,0, 0,0,0,0]"/> <!-- duration of each tone in units of 10 milli-seconds. Poor naming!!! -->
    <param name="tone_powers"         value="[60,  60,  60,  60,  0,0,0,0, 0,0,0,0]"/> <!-- max is 255 -->

    <param name="dt_threshold_ns"       value="150000"/>      <!-- during start up, ignore inter-commutation times less than this val, probably noise -->
    <param name="max_dt_ns"             value="2500000"/>     <!-- min and max values for time between two commutations. these are used as caps -->
    <param name="min_dt_ns"             value="10000"/>
    <param name="dt_bootstrap_ns"       value="2000000"/>     <!-- filter bootstrap value for commutation dt during start up -->

    <param name="spinup_stall_dt_ns"    value="6000000"/>     <!-- during spin-up, if no zero crossing is not detected for this amount of time, motor is considered stalled -->
    <param name="spinup_stall_check_ns" value="30000000"/>    <!-- time after beginning of spinup to start checking for spinup stall -->

    <param name="alignment_time_ns"     value="0"/>           <!-- alignment time before spin-up -->
    <param name="timing_advance"        value="0"/>
    <param name="sense_advance"         value="0"/>

    <param name="demag_timing"          value="0"/>    <!-- unused -->

  </TuneParams>
</EscParameters>

[Alex Kushleyev]

@John-Nomikos-0

Thank you. OK, here is the big difference that is most likely the reason for your behavior. In your new params you used kp, ki, max_kpe, max_kpi terms set to zero, but old params have nonzero:

<param name="kp"                  value="130"/>    <!-- RPM controller proportional gain -->
<param name="ki"                  value="30"/>     <!-- RPM controller proportional gain -->
<param name="max_kpe"             value="300"/>    <!-- maximum proportional erorr term (max is 999) -->
<param name="max_kie"             value="100"/>    <!-- maximum integral error term (max is 999) -->
<param name="max_rpm_delta"       value="1200"/>    <!-- cap for maximum rpm error used in RPM controller -->

Considering that you have not calibrated the ESC for your motor and propeller (meaning that the feed-forward term in the RPM controller is wrong), if you also disable the feedback terms of the RPM controller (kp, ki) then the behavior of the ESC will be completely off from what is desired. Even with incorrect feed forward term (pwm_vs_rpm_curve terms), if feedback terms are enabled, they can compensate, to some degree, but with the feedback terms set to zero, there is no hope for the ESC to turn the motor at the desired speed.

You can keep using the old firmware and params if that works for you, but if you wanted to switch to latest ESC firmware and params, then i suggest trying to use the same non-zero RPM controller params.

Alex