Topics

Compass udrc kernel panic

Bernard Pidoux
 

Hi Annaliese,

Let me introduce myself first. I have been maintaining ham radio application under Linux for two decades, linfbb and FPAC. I am also contributing occasionally to Linux net modules AX25 and rose. I have a couple of raspberry pi with UDRC II hats running compass distro with Direwolf APRS and at the same time AX25 packet radio network applications. Everything is running flawlessly with last updates.
I recently experienced kernel panics with compass Linux distro with kernel 4.14.79-v7+ as soon as opening a Chromium window.
Kernel panic does not occur if I rmmod all snd and udrc modules before activating Chromium.
After removing all sound and udrc module, modprobe udrc does not reload all needed modules for aplay - l to find udrc card.
I attach one of the photo I took from screen while kernel panic. It shows a null pointer related to bcm2835.
When investigating kernel panic I noticed a few dmesg messages:
sdhost-bcm2835 3f202000.mmc could not get clk, deferring probe.
udrc: loading out-of-tree module taints kernel.
snd-udrc soc:sound: ASoC: CODEC DAI tlv320aic32x4-hifi not registered -will retry
snd-udrc soc:sound: snd_soc_register_card() failed: -517
If you need I can send you more details and screen pictures.

Regards,

Dr Bernard Pidoux
F6BVP

Basil Gunn
 

Hi Bernard,

Understand that compass is raspbian is Debian. Please ssh into your
target machine & cut & paste text rather than attaching screen shots,
then you can show us more information from the panic.

The kernel panic from your screen shot is from kernel file:
net/ax25/ax25_addr.c in routine ax25cmp() which compares two ax25
addresses one of which is null.

It looks to me like this is occurring when FPAC is binding to device
rose0 but it's difficult to parse the screen shot you attached.

Anna might comment on the messages coming out of soc:sound. Eventually
the tlv320aic code registers with i2s

snd-udrc soc:sound: tlv320aic32x4-hifi <-> 3f203000.i2s mapping ok

From the information you have supplied I believe this is a Rose ax.25
problem and not a compass/udrc codec problem.

/Basil

Bernard Pidoux <bernard.f6bvp@...> writes:

Hi Annaliese,

Let me introduce myself first. I have been maintaining ham radio
application under Linux for two decades, linfbb and FPAC. I am also
contributing occasionally to Linux net modules AX25 and rose. I have a
couple of raspberry pi with UDRC II hats running compass distro with
Direwolf APRS and at the same time AX25 packet radio network
applications. Everything is running flawlessly with last updates.

I recently experienced kernel panics with compass Linux distro with
kernel 4.14.79-v7+ as soon as opening a Chromium window. Kernel panic
does not occur if I rmmod all snd and udrc modules before activating
Chromium. After removing all sound and udrc module, modprobe udrc
does not reload all needed modules for aplay - l to find udrc card. I
attach one of the photo I took from screen while kernel panic. It
shows a null pointer related to bcm2835. When investigating kernel
panic I noticed a few dmesg messages: sdhost-bcm2835 3f202000.mmc
could not get clk, deferring probe. udrc: loading out-of-tree module
taints kernel. snd-udrc soc:sound: ASoC: CODEC DAI tlv320aic32x4-hifi
not registered -will retry snd-udrc soc:sound: snd_soc_register_card()
failed: -517 If you need I can send you more details and screen
pictures.

Regards,

Dr Bernard Pidoux
F6BVP

Annaliese McDermond
 

On Dec 6, 2018, at 9:48 AM, Basil Gunn <@basil860> wrote:


Anna might comment on the messages coming out of soc:sound. Eventually
the tlv320aic code registers with i2s

snd-udrc soc:sound: tlv320aic32x4-hifi <-> 3f203000.i2s mapping ok
Indeed the mapping fails a few times before it succeeds. This is because of driver dependency ordering. Many times the tlv320aic32x4 driver doesn’t look quickly enough for the other pieces of the sound driver. This is why the framework retries, because module dependencies aren’t always the greatest in Linux.

Bernard Pidoux <bernard.f6bvp@...> writes:

Hi Annaliese,

I recently experienced kernel panics with compass Linux distro with
kernel 4.14.79-v7+ as soon as opening a Chromium window. Kernel panic
does not occur if I rmmod all snd and udrc modules before activating
Chromium. After removing all sound and udrc module, modprobe udrc
does not reload all needed modules for aplay - l to find udrc card. I
attach one of the photo I took from screen while kernel panic. It
shows a null pointer related to bcm2835.
There’s no relation really to BCM2835. The BCM2835 is the Broadcom System on a Chip part number of the SoC on the Raspberry Pi. The Oops message is telling you that it’s running on a Raspberry Pi. It has no real relation to the cause of the panic.

When investigating kernel
panic I noticed a few dmesg messages:
sdhost-bcm2835 3f202000.mmc could not get clk, deferring probe.
This is the Raspberry Pi’s MMC interface on the MMC. It’s just commenting that whatever clock it’s running isn’t ready yet, and it’ll load the driver later. This is a normal condition on the Pi during boot.

udrc: loading out-of-tree module taints kernel.
This just means that we compiled the udrc kernel module “out of tree” instead of “in tree.” That means we don’t have the full Linux kernel source, just the necessary headers. It’s not a part of the kernel itself. This is a normal condition.


snd-udrc soc:sound: ASoC: CODEC DAI tlv320aic32x4-hifi not registered -will retry
snd-udrc soc:sound: snd_soc_register_card() failed: -517
See above.

If you need I can send you more details and screen
pictures.
I think Basil has a sound analysis here that you’re seeing a panic in the ax25 stack somewhere. I’d have to investigate what the changes to the ax25 stack were between kernel versions and try to bisect it.

If you’d like to know my offhand guess as to what’s happening, I’d guess that Chromium is trying to do a broadcast to all interfaces for some reason. Once you load the ax25 stack, the packet modem becomes a network interface. This broadcast packet is probably triggering a bug in the ax25 stack due to it having some sort of freaky address (or more accurately no address) by the time it gets there. But this is just a wild guess from hearing the symptoms.

Other than the fact it’s saying that the tlv320aic32x4 driver is loaded at the time of panic, there’s nothing here that really points to the udrc; just to the ax25 stack. You might want to poke at the kernel ax25 folks and they may be able to illuminate more. Or Basil grubs around in that chunk of the kernel more than I and may have some knowledge to illuminate the situation.


Regards,

Dr Bernard Pidoux
F6BVP


--
Annaliese McDermond (NH6Z)
Xenotropic Systems
mcdermj@...

Curt, WE7U
 

On Thu, 6 Dec 2018, Annaliese McDermond wrote:
If you?d like to know my offhand guess as to what?s happening, I?d guess that Chromium is trying to do a broadcast to all interfaces for some reason. Once you load the ax25 stack, the packet modem becomes a network interface. This broadcast packet is probably triggering a bug in the ax25 stack due to it having some sort of freaky address (or more accurately no address) by the time it gets there. But this is just a wild guess from hearing the symptoms.
Check Samba as well if it's loaded and running. It's a known contributor to "talk to all interfaces" problems.

--
Curt, WE7U. http://we7u.wetnet.net
U.S. Weather Alerts: Firenet.aprs2.net, port 14580, filter "t/n e/WE7U-WX"
Coordinate Converter (Android): http://www.sarguydigital.com

f6bvp
 

I would like to thank all who sent a reply to my kernel panic message.

Basil Gunn contributed a lot in analyzing the screen dump :
" The kernel panic from your screen shot is from kernel file:
net/ax25/ax25_addr.c in routine ax25cmp() which compares two ax25
addresses one of which is null." he wrote.
His message reminded me an old bug I found in february 2016 in rose
ax.25 packet radio protocole, when ax25cmp has a NULL argument,
triggering a kernel panic.
https://marc.info/?l=linux-netdev&m=146866282201792&w=2
https://marc.info/?l=linux-hams&m=147118203413414&w=2
Thus I patched the offending function and everything turned back to
excellent behavior.
I guess that Annaliese analysis is correct in that association of
Chromium and rose unraveled the old bug, during tasks swap (?).
I now have to convince netdev authority to accept the simple patch.
A couple of years ago it did not. However now that kernel panic occurs
systematically when opening Chromium the patch can be more easily accepted.

https://marc.info/?l=linux-netdev&m=145633324325000&w=2
https://marc.info/?l=linux-netdev&m=145643820631007&w=2
https://marc.info/?l=linux-netdev&m=145704258127811&w=2
https://marc.info/?l=linux-hams&m=147118203413414&w=2

If not, I am thinking of adopting dkms for rose module to be
automatically re installed each time kernel updates. A package with dkms
rose module could be built, turning around "official" kernel distro.

Thanks again to you all.

Bernard


Le 07/12/2018 à 15:35, Curt, WE7U a écrit :

On Thu, 6 Dec 2018, Annaliese McDermond wrote:

If you?d like to know my offhand guess as to what?s happening, I?d
guess that Chromium is trying to do a broadcast to all interfaces for
some reason.  Once you load the ax25 stack, the packet modem becomes a
network interface.  This broadcast packet is probably triggering a bug
in the ax25 stack due to it having some sort of freaky address (or
more accurately no address) by the time it gets there.  But this is
just a wild guess from hearing the symptoms.
Check Samba as well if it's loaded and running. It's a known contributor
to "talk to all interfaces" problems.

Basil Gunn
 

Curt, WE7U <curt.we7u@...> writes:

On Thu, 6 Dec 2018, Annaliese McDermond wrote:

If you?d like to know my offhand guess as to what?s happening, I?d
guess that Chromium is trying to do a broadcast to all interfaces for
some reason. Once you load the ax25 stack, the packet modem becomes
a network interface. This broadcast packet is probably triggering a
bug in the ax25 stack due to it having some sort of freaky address
(or more accurately no address) by the time it gets there. But this
is just a wild guess from hearing the symptoms.
Check Samba as well if it's loaded and running. It's a known
contributor to "talk to all interfaces" problems.
When I bring up an AX.25 machine I always install iptables & block the
following 3 addresses on both ax0 & ax1 interfaces. I noticed these
addresses being broadcast/multicast while looking at a traffic spy of my
RF interfaces.

iptables -A OUTPUT -o ax0 -d 224.0.0.22 -p igmp -j DROP
iptables -A OUTPUT -o ax0 -d 224.0.0.251 -p udp -m udp --dport 5353 -j DROP
iptables -A OUTPUT -o ax0 -d 239.255.255.250 -p udp -m udp -j DROP

iptables -A OUTPUT -o ax1 -d 224.0.0.22 -p igmp -j DROP
iptables -A OUTPUT -o ax1 -d 224.0.0.251 -p udp -m udp --dport 5353 -j DROP
iptables -A OUTPUT -o ax1 -d 239.255.255.250 -p udp -m udp -j DROP

224.0.0.22 is used for IGMPv3 (multicast) protocol
224.0.0.251 is Bonjour/mDNS request coming out of the Avahi daemon
239.255.255.250 is Simple Service Discovery Protocol (SSDP) for uPNP
detection

I agree with Curt that Samba & dropbox will broadcast on all
network addresses but you can fix both of these with proper
configuration of the app.

/Basil