RNDIS Host not working

[OliveBen]

Hello!

We are trying to daisy chain multiple BRK devices together via ethernet over usb.

When connecting one BRK to the PC (Ubuntu 20.04) everything works fine. My PC acts as a rndis host and we can do ethernet over usb, with ssh. So far so good. Now we would like to turn the BRK board itself also into a rndis_host so that we can attach a second BRK to the first BRK’s usb, effectively forming a chain of linked devices which should ideally all be reachable from the host PC (their client IP addresses are configured appropriately).

 

To do this I added rndis_host to the bootfs patches and rebuilt the kernel. On the device /proc/config.gz shows that the correct config wa used to build the kernel.

However, rndis_host is not shown in lsmod. Neither are other relevant kernel modules like usbnet or cdc_ether. ‘modprobe rndis_host’ retuns no errors, no nothing.

Furthermore, when connecting the second BRK to the first via USB I get these messages in dmesg:

usb 2-1: set a_alt_hnp_support failed: -32

usb 2-1: device descriptor read/64, error -110

usb 2-1: device not accepting address 4, error -110 Is ther a way to make this work?

 

Thank you very much Benjamin

• This topic was modified 2 years ago by OliveBen.

[OliveBen]

A few notes:

the second BRK (let’s call it BRK B), which I connect to the first one (BRK A), is working for sure, bc I can use it normally when connecting to the host PC. So the issue must be with BRK A’s rndis_host capabilities.

the dmesg errors also show up on BRK B when I connect a it the other way around. (PC -> BRK B -> BRK A). So they seem to be independent of the rndis_host kernel module.

Any help would be greatly appreciated.

Thanks!

[coloradocarlos]

What would be helpful would be to describe your hardware and cabling configuration in more detail. The BRK board has a micro-USB port that is designed to work as a USB client. There are no other USB ports on the OSD32MP1-BRK board unless you added something to the board (or Octavo has something I am not aware of).

What is not clear either is if you trying to use the OTG role-switching “USB gadget” feature on the BRK board.

You may want to first test OTG host-mode using an OTG cable on another device, for example, HID keyboard or USB flash drive. OTG cable informs the hardware and OS that you want to place the USB in the host role.

Concerning loadable modules, it appears you compiled the module into the kernel and so it may not appear as a loadable module. If you are using OpenSTLinux, verify the module exists:

find /lib/modules/5.10.10/kernel/drivers/usb/gadget -name usb*

Carlos

 

[OliveBen]

Hi coloradocarlos and thanks for the swift response!

so we connect the BRK to the host PC via the regular micro USB connector. Then the BRK has a USB_DM1 and USB_DP1 pin. Together with a GND and +VIN pin we soldered a female USB connector to connect other hardware. It works like a regular USB. I tried connecting other USB devices like cameras before and it worked without a problem. So we don’t need to make use of OTG.

Instead of switching between host and client mode, we want the board to be both at the same time: act as a peripheral to the host PC through its main micro USB connector and act as a host to another BRK through the USB_DM1 abd USB_DP1 pins.

Does that make sense?

Best

Ben

[coloradocarlos]

OK, so you modified the BRK hardware and the USB I assume is a USB Type-A. I would need to consult the STM documentation on the pins used for USB and specifically to support OTG mode detect.

This is what I have (I also have a CD-ROM gadget configured):

root@myboard:~# lsmod | grep rndis usb_f_rndis 24576 2 u_ether 20480 1 usb_f_rndis libcomposite 49152 12 usb_f_mass_storage,usb_f_rndis

1 2 3 4

root@myboard:~# lsmod | grep rndis usb_f_rndis            24576  2 u_ether                20480  1 usb_f_rndis libcomposite           49152  12 usb_f_mass_storage,usb_f_rndis

On the device board, restart the gadget service:

systemctl restart usbotg-config

1	systemctl restart usbotg-config

On the host board you should see messages in dmesg related to the device board bringing up rndis. This is where it gets a little tricky as I don’t know what to expect with OpenSTLinux. But you need to bring up an interface on the host and I highly recommend to set things up so packets are routed on different subnets (don’t rely on the default 192.168.7.x).

ifconfig enx123456789 192.168.8.2 up

1	ifconfig enx123456789 192.168.8.2 up

Where “12345…” is a MAC address of the USB network. Try using ifconfig to see what interfaces are configured. If none are configured, then rndis “host” is not detecting the “device”. lsusb -v might help.

The other key thing is you will likely need to modify /sbin/stm32_usbotg_eth_config.sh on the device to assign an IP address on a different subnet. In other words, the device in the middle is acting as a gateway router.

[OliveBen]

Thank you Carlos,

Yes it is a USB Type-A connector.
I do have the same modules enabled as you have (among others of course) and the issue persists.

One interesting lead might be this:

When I connect the BRK to my host PC, this is dmesg from the host:

[21293.581039] usb 1-2: new high-speed USB device number 9 using xhci_hcd
[21293.729697] usb 1-2: New USB device found, idVendor=1d6b, idProduct=0104, bcdDevice= 1.00
[21293.729704] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[21293.729707] usb 1-2: Product: STM32MP1
[21293.729710] usb 1-2: Manufacturer: STMicroelectronics
[21293.729712] usb 1-2: SerialNumber: 002700203239511337383434
[21293.738493] rndis_host 1-2:1.0 eth0: register ‘rndis_host’ at usb-0000:00:14.0-2, RNDIS device, 58:09:4b:39:9a:e7
[21293.803092] rndis_host 1-2:1.0 enx58094b399ae7: renamed from eth0

 

This is how things should work. The host identifies the USB device and then spins up rndis_host

However, when I connect the peripheral BRK to the host BRK, this is the dmesg from the host BRK:

[  296.083633] usb 2-1: new high-speed USB device number 2 using ehci-platform
[  296.285007] usb 2-1: Dual-Role OTG device on non-HNP port
[  296.289115] usb 2-1: set a_alt_hnp_support failed: -32
[  296.463623] usb 2-1: new high-speed USB device number 3 using ehci-platform
[  301.663615] usb 2-1: device descriptor read/64, error -110
[  317.023670] usb 2-1: device descriptor read/64, error -110

 

Note the second line ‘Dual-Role OTG device on non-HNP port’…
This makes me belive that before even running rndis, the USB lane on the host (pins USB_DM1 and USB_DP1) does not support HNP and thus can’t identify the peripheral as such.

Now the question would be how to enable HNP for this USB connector.

Do you think restarting usbotg-config on the peripheral would make a change to this HNP problem?

[OliveBen]

Concerning the kernel modules. Yes I baked them into the kernel. Here’s the info you suggested. Note that rndis is missing from the drivers/net/usb directory

 

[OliveBen]

Regarding the kernel modules: yes I baked them into the kernel directly. I assumed they would still show up in lsmod. Here’s the info you requested. Note that rndis_host is missing from drivers/net/usb.

user@localhost:~$ find /lib/modules/5.10.10/kernel/drivers/usb/gadget -name usb*
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_ncm.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_uac1_legacy.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_hid.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_uvc.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_midi.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_fs.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_rndis.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_uac2.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_printer.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_acm.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_serial.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_uac1.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_eem.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_ecm.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko
/lib/modules/5.10.10/kernel/drivers/usb/gadget/function/usb_f_obex.ko

user@localhost:~$ find /lib/modules/5.10.10/kernel/drivers/net/usb/
/lib/modules/5.10.10/kernel/drivers/net/usb/
/lib/modules/5.10.10/kernel/drivers/net/usb/lan78xx.ko
/lib/modules/5.10.10/kernel/drivers/net/usb/r8152.ko

 

• This reply was modified 2 years ago by OliveBen.

[OliveBen]

Thank you Carlos,

Yes it is a USB Type-A connector.
I do have the same modules enabled as you have (among others of course) and the issue persists.

One interesting lead might be this:

When I connect the BRK to my host PC, this is dmesg from the host:

[21293.581039] usb 1-2: new high-speed USB device number 9 using xhci_hcd
[21293.729697] usb 1-2: New USB device found, idVendor=1d6b, idProduct=0104, bcdDevice= 1.00
[21293.729704] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[21293.729707] usb 1-2: Product: STM32MP1
[21293.729710] usb 1-2: Manufacturer: STMicroelectronics
[21293.729712] usb 1-2: SerialNumber: 002700203239511337383434
[21293.738493] rndis_host 1-2:1.0 eth0: register ‘rndis_host’ at usb-0000:00:14.0-2, RNDIS device, 58:09:4b:39:9a:e7
[21293.803092] rndis_host 1-2:1.0 enx58094b399ae7: renamed from eth0

 

This is how things should work. The host identifies the USB device and then spins up rndis_host

However, when I connect the peripheral BRK to the host BRK, this is the dmesg from the host BRK:

[  296.083633] usb 2-1: new high-speed USB device number 2 using ehci-platform
[  296.285007] usb 2-1: Dual-Role OTG device on non-HNP port
[  296.289115] usb 2-1: set a_alt_hnp_support failed: -32
[  296.463623] usb 2-1: new high-speed USB device number 3 using ehci-platform
[  301.663615] usb 2-1: device descriptor read/64, error -110
[  317.023670] usb 2-1: device descriptor read/64, error -110

 

Note the second line ‘Dual-Role OTG device on non-HNP port’…
This makes me belive that before even running rndis, the USB lane on the host (pins USB_DM1 and USB_DP1) does not support HNP and thus can’t identify the peripheral as such.

Now the question would be how to enable HNP for this USB connector.

Do you think restarting usbotg-config on the peripheral would make a change to this HNP problem?

[coloradocarlos]

Take a second look at the device trees for the modified BRK board with the Type-A female USB. Did you generate them using Cube MX? The default device tree from Octavo will probably not work because your “new” hardware differs than a stock BRK board. Check the USB nodes and pinmux assignments in the device tree for the kernel, uboot, and tf-a. The RED board has a USB host and it should serve as an example.

You mentioned that the host USB works with a flash drive or keyboard, but not a USB cable to the peripheral BRK. This suggests that HNP could be failing because both devices are supplying power at the same time and can’t “negotiate” who is the host for OTG. So take a look at the power rails too.

[mkleox]

Hi Coloradocarlos,

Thanks for helping us with this issue.

Here is the schematic of the connected hardware:

 

 

Attachments:

1. 

   Connection.jpg
   

[mkleox]

When we connect a keyboard, USB flash, or a camera instead of the BRK-B board to the USB connector of the BRK-A board, it works, however, it seems that the RNDIS-host driver is not loaded on the BRK-A board, that’s why when we connect the BRK-B to BRK-A, it doesn’t recognize the board.

[Aedan Cullen]

There are two problems here. First, we need to have enumeration on the bus before worrying about RNDIS, but we are not finishing enumeration. (The a_alt_hnp_support failed: -32 error comes from usb_enumerate_device_otg in drivers/usb/core/hub.c.) Lack of RNDIS host support on BRK A may be a problem, but it is a separate problem to worry about after we make it through enumeration.

Investigating that error, I believe the dwc2 driver may have incorrect HNP/SRP attributes in its OTG descriptor when used as a gadget device in kernel 5.10 (that’s the condensed summary of the mess.)

First, verify that you don’t see BRK B in lsusb from BRK A, even though it is connected. This confirms that the enumeration problem exists.

Now, try disabling OTG on the microUSB connector of BRK B by setting its role to a peripheral. The original 5.10 devicetree from Octavo contains this:

&usbotg_hs{

u-boot,dm-pre-reloc;

status = "okay";

phys = <&usbphyc_port1 0>;

phy-names = "usb2-phy";

};

Try changing it as follows and rebuilding the kernel:

&usbotg_hs{

u-boot,dm-pre-reloc;

status = "okay";

phys = <&usbphyc_port1 0>;

phy-names = "usb2-phy";

dr_mode = "peripheral";

};

The above is successful if you no longer see the a_alt_hnp_support error. Unfortunately I’m away from my drawer of Octavo hardware until next semester starts so I can’t test this and it’s very much a shot in the dark. If this does eliminate that error and get BRK B to appear in lsusb, then I will post a full explanation of what was happening and we can proceed to worrying about rndis_host.

If that doesn’t get you anywhere, you could also try adding usb-role-switch; and role-switch-default-mode = "peripheral"; to that same devicetree node.

[coloradocarlos]

The schematic is very helpful and a useful visual. This allows us to review chapters 60, 61, and 62 of RM0436.

BRK-A is not bringing up RNDIS because it doesn’t get that far in the initialization. Try “lsusb -v | grep -i gadget” and see if the “Linux Foundation Multifunction Composite Gadget” shows up as one of the devices that are detected. If the Gadget does not come up, then none of the supported “gadgets” like RNDIS will not work. You can also try “lsbusb -t” and see if the rndis_host driver is listed in the classes.

How is BRK-B powered? Is it receiving power from BRK-A USB Type-A or an external source on VIN?

[Aedan Cullen]

There are two problems here. First, we need to have enumeration on the bus before worrying about RNDIS, but we are not finishing enumeration. (The set a_alt_hnp_support failed: -32 error comes from usb_enumerate_device_otg in drivers/usb/core/hub.c.) Lack of RNDIS host support on BRK A may be a problem, but it is a separate problem to worry about after we make it through enumeration.

 

Investigating that error, I believe the dwc2 driver may have incorrect HNP/SRP attributes in its OTG descriptor when used as a gadget device in kernel 5.10 (that’s the condensed summary of the mess.)

First, verify that you don’t see BRK B in lsusb from BRK A, even though it is connected. This confirms that the enumeration problem exists.

Now, try disabling OTG on the microUSB connector of BRK B by setting its role to a peripheral. The original 5.10 devicetree from Octavo contains this:

&usbotg_hs{ u-boot,dm-pre-reloc; status = "okay"; phys = <&usbphyc_port1 0>; phy-names = "usb2-phy"; };

1 2 3 4 5 6

&usbotg_hs{ u-boot,dm-pre-reloc; status = "okay"; phys = <&usbphyc_port1 0>; phy-names = "usb2-phy"; };

Try changing it as follows and rebuilding the kernel:

&usbotg_hs{ u-boot,dm-pre-reloc; status = "okay"; phys = <&usbphyc_port1 0>; phy-names = "usb2-phy"; dr_mode = "peripheral"; };

1 2 3 4 5 6 7

&usbotg_hs{ u-boot,dm-pre-reloc; status = "okay"; phys = <&usbphyc_port1 0>; phy-names = "usb2-phy"; dr_mode = "peripheral"; };

This is successful if you no longer see the a_alt_hnp_support error. Unfortunately I’m away from my drawer of Octavo hardware until next semester starts so I can’t test this and it’s very much a shot in the dark. If this does eliminate that error and get BRK B to appear in lsusb, then I will post a full explanation of what was happening and we can proceed to worrying about rndis_host.

If that doesn’t get you anywhere, you could also try adding usb-role-switch; and role-switch-default-mode = "peripheral"; to that same devicetree node.

[Aedan Cullen]

There are two problems here. First, we need to have enumeration on the bus before worrying about RNDIS, but we are not finishing enumeration. (The a_alt_hnp_support failed: -32 error comes from usb_enumerate_device_otg in drivers/usb/core/hub.c.) Lack of RNDIS host support on BRK A may be a problem, but it is a separate problem to worry about after we make it through enumeration.

Investigating that error, I believe the dwc2 driver may have incorrect HNP/SRP attributes in its OTG descriptor when used as a gadget device in kernel 5.10 (that’s the condensed summary of the mess.)

First, verify that you don’t see BRK B in lsusb from BRK A, even though it is connected. This confirms that the enumeration problem exists.

Now, try disabling OTG on the microUSB connector of BRK B by setting its role to a peripheral. The original 5.10 devicetree from Octavo contains this:

&usbotg_hs{ u-boot,dm-pre-reloc; status = "okay"; phys = <&usbphyc_port1 0>; phy-names = "usb2-phy"; };

1 2 3 4 5 6

&usbotg_hs{     u-boot,dm-pre-reloc;     status = "okay";     phys = <&usbphyc_port1 0>;     phy-names = "usb2-phy"; };

Try changing it as follows and rebuilding the kernel:

&usbotg_hs{ u-boot,dm-pre-reloc; status = "okay"; phys = <&usbphyc_port1 0>; phy-names = "usb2-phy"; dr_mode = "peripheral"; };

1 2 3 4 5 6 7

&usbotg_hs{     u-boot,dm-pre-reloc;     status = "okay";     phys = <&usbphyc_port1 0>;     phy-names = "usb2-phy";     dr_mode = "peripheral"; };

This is successful if you no longer see the a_alt_hnp_support error. Unfortunately I’m away from my drawer of Octavo hardware until next semester starts so I can’t test this and it’s very much a shot in the dark. If this does eliminate that error and get BRK B to appear in lsusb, then I will post a full explanation of what was happening and we can proceed to worrying about rndis_host.

If that doesn’t get you anywhere, you could also try adding usb-role-switch; and role-switch-default-mode = "peripheral"; to that same devicetree node.

[Aedan Cullen]

There are two problems here. First, we need to have enumeration on the bus before worrying about RNDIS, but we are not finishing enumeration. (The a_alt_hnp_support failed: -32 error comes from usb_enumerate_device_otg in drivers/usb/core/hub.c.) Lack of RNDIS host support on BRK A may be a problem, but it is a separate problem to worry about after we make it through enumeration.

Investigating that error, I believe the dwc2 driver may have incorrect HNP/SRP attributes in its OTG descriptor when used as a gadget device in kernel 5.10 (that’s the condensed summary of the mess.)

First, verify that you don’t see BRK B in lsusb from BRK A, even though it is connected. This confirms that the enumeration problem exists.

Now, try disabling OTG on the microUSB connector of BRK B by setting its role to a peripheral. The original 5.10 devicetree from Octavo contains this:

&usbotg_hs{

u-boot,dm-pre-reloc;

status = "okay";

phys = <&usbphyc_port1 0>;

phy-names = "usb2-phy";

};

Try changing it as follows and rebuilding the kernel:

&usbotg_hs{

u-boot,dm-pre-reloc;

status = "okay";

phys = <&usbphyc_port1 0>;

phy-names = "usb2-phy";

dr_mode = "peripheral";

};

The above is successful if you no longer see the a_alt_hnp_support error. Unfortunately I’m away from my drawer of Octavo hardware until next semester starts so I can’t test this and it’s very much a shot in the dark. If this does eliminate that error and get BRK B to appear in lsusb, then I will post a full explanation of what was happening and we can proceed to worrying about rndis_host.

If that doesn’t get you anywhere, you could also try adding usb-role-switch; and role-switch-default-mode = "peripheral"; to that same devicetree node.

• This reply was modified 1 year, 11 months ago by Aedan Cullen.
• This reply was modified 1 year, 11 months ago by Aedan Cullen.

[OliveBen]

Hello Aedan and thanks a lot for the thorough explanation. It works! I changed the device tree as you suggested and now. I can see the new gadget in lsusb.
this is the new dmesg output:

[ 1567.864660] usb 2-1: new high-speed USB device number 2 using ehci-platform
[ 1568.173658] rndis_host 2-1:1.0 eth0: register ‘rndis_host’ at usb-5800d000.usbh-ehci-1, RNDIS device, d8:de:9b:83:8d:35
[ 1568.193069] usbcore: registered new interface driver rndis_host
[ 1568.260294] rndis_host 2-1:1.0 enxd8de9b838d35: renamed from eth0

And this is the new lsusb output:

Bus 002 Device 002: ID 1d6b:0104 Linux Foundation Multifunction Composite Gadget
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Now I am not sure how to proceed. I followed coloradocarlos suggestion to bring up a new interface using.

ifconfig enxd8de9b838d3510.42.0.7up However when I SSH into that new ip address I am in fact accessing the host BRK A again instead of acessing the peripheral BRK B. Could you kindly provide me a hint how to proceed? Best Benjamin

 

[OliveBen]

Apologies  I made a mistake there. The command I tried is

ifconfig enxd8de9b838d35 10.42.0.8 up

 

I verified that the MAC address is of BRK B. The ip address I used is what is configured as a static IP address in BRK B

[Neeraj Dantu]

Banjamin,

In order to get an SSH connection over to the peripheral BRK, you will need to know the IP address of that BRK. You can bring up a console on the peripheral BRK by connecting to UART4 via a USB-UART cable, and then look at the output of ifconfig there. If you do not have an IP address, you can assign a static IP on that board and then you will need to use that IP address to SSH into that board from the host BRk board.

Best,

Neeraj

[Aedan Cullen]

Because of the unusual “daisy-chain” configuration, we will need the first BRK in the chain (“BRK A”) to route packets through itself to the other device. A bridge interface is probably the best way to accomplish this. (Without a bridge, packets received at the first interface on BRK A will hit a dead end since this routing to BRK B will not happen automatically.) Linux treats a bridge as a network switch with other interfaces attached, and there will be simply a single IP address assigned to the bridge, which should serve as the address for packets to that particular BRK board. There are many guides on interface bridging online, but the general idea is this:

ip link add name br0 type bridge

ip link set dev br0 up

Then make sure both your “upstream” interface (which I’ll call eth0 here) and your “downstream” interface (which I’ll call eth1) are up and running (e.g. you can see RUNNING in ifconfig). Then add both to the bridge:

ip link set eth0 master br0

ip link set eth1 master br0

Then assign the bridge an IP address so that it will receive packets destined for BRK A itself:

ip address add dev br0 192.168.7.1/24

This should allow packets not for 192.168.7.1 to be routed to the other RNDIS interface and to BRK B. The IP address configuration of BRK B will matter too. For resolving issues it would be helpful if you post your ifconfig output on BRK A to provide a better picture of what the situation looks like.

Note: though theoretically this USB “daisy chain” idea could indeed be used with many devices, I would caution against using it where reliability is important or a consistent low-latency interconnect is needed. In some environments prone to unanticipated ESD events and excessive EMI from motors (such as robotics competitions), systems based on many individual USB 2.0 connections have proved extremely troublesome in the past and have been replaced with RS-485.

[OliveBen]

Hi Aedan and thanks a lot again. We did finally make it work! Thanks a lot, we followed the approach of the bridge, but used bridge-tools.

 

To be precise I wrote a bash script that is executed at boot time. It removes the IP address from the USB0 interface, sets up a new bridge br0 under the same IP address  and links the USB0 interface to this bridge. So by default we can address the device normally.

Then I wrote a udev rule that checks for new USB connections of the device STM32MP1. If it detects one, it links the new device via its MAC address to br0.

 

Works like a charm. Again massive kudos for the support!

[Author]

Posts