Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mt7615 firmware init failed after soft reboot (reset required?) #316

Open
dengqf6 opened this issue Sep 30, 2019 · 19 comments
Open

mt7615 firmware init failed after soft reboot (reset required?) #316

dengqf6 opened this issue Sep 30, 2019 · 19 comments

Comments

@dengqf6
Copy link

dengqf6 commented Sep 30, 2019

@nbd @chunkeey @LorenzoBianconi
SoC is IPQ4019.

After a soft reboot, MT7615 no longer works

mt7615e 0000:01:00.0: enabling device (0140 -> 0142)
mt7615e 0000:01:00.0: Firmware is not ready for download
mt7615e: probe of 0000:01:00.0 failed with error -5

I checked the val in mt7615_load_firmware

if (val != FW_STATE_FW_DOWNLOAD) {
and it is already 7 (FW_STATE_CR4_RDY)

It seems that either the PCIe driver does not power cycle the PCIe interface after reboot, or the mt76 driver does not warm reset the mt7615 chip.

Update: If I unload the driver manually by rmmod mt7615e, it will work after the reboot

@chunkeey
Copy link
Member

chunkeey commented Oct 2, 2019

It seems that either the PCIe driver does not power cycle the PCIe interface after reboot, or the mt76 driver does not warm reset the mt7615 chip.

That wouldn't surprise me. A lot of ath10k chips have problems after being PCIe reseted... if done inproperly they are taking down the system with it...

That said, there's also patches like this that might not have trickled down (ye):

https://patchwork.kernel.org/patch/10966623/

So please check if this patch still applies to your case.

@dengqf6
Copy link
Author

dengqf6 commented Oct 3, 2019

@chunkeey The patch has already been backported to 4.19
gregkh/linux@6b71c62
and it does not help

@tinxx
Copy link

tinxx commented Nov 14, 2020

I have added a script as a workaround on my device that will automatically execute rmmod mt7615e on shutdown/reboot.

Create a file /etc/init.d/mt7615workaround with content:

#!/bin/sh /etc/rc.common

STOP=52  
       
stop() {
  rmmod mt7615e
} 

Then link it into /etc/rc.d by executing cd /etc/rc.d && ln -s ../init.d/mt7615workaround K52mt7615wa.

@aparcar
Copy link
Member

aparcar commented Mar 30, 2021

Any updates on this? I'm facing a similar problem with a Linksys E8450 aka Belkin AXRT3200:

Tue Mar 30 00:01:16 2021 kern.debug kernel: [ 1596.726529] mt7915e 0000:01:00.0: assign IRQ: got 144
Tue Mar 30 00:01:16 2021 kern.debug kernel: [ 1596.731735] mt7915e 0000:01:00.0: enabling bus mastering
Tue Mar 30 00:01:20 2021 kern.err kernel: [ 1600.565394] mt7915e 0000:01:00.0: Firmware is not ready for download
Tue Mar 30 00:01:20 2021 kern.warn kernel: [ 1600.572694] mt7915e: probe of 0000:01:00.0 failed with error -5

@dolhastelian
Copy link

Is there any update on this topic ? I am facing this issue on IpTime A8004T device.
After first install of kernel and sysupgrade I can configure and use both 2.4 and 5 G radio devices but after reset only 5G radio works. I tried to play with some parameters like channel change, country code change, serialize option on but nothing seems to bring back the device to life. I am using 21.02.0-rc4 (r16256-2d5ee43dc6)

@wojciech-z
Copy link

I had the same problem with Totolink A7000R. 2.4Ghz radio didn't even start up. 5GHz was working fine.
I've tried (soft)rebooting several times with no result. Finally i did a power cycle and this time 2.4GHz worked.
I've followed @tinxx instructions and so far it works, even after rebooting (both software reboot and poweroff).

Many thanks to you, @tinxx, and of course everyone involved in OpenWrt 🎉

@HausnerR
Copy link

Any news? Upgraded router to newest OpenWrt and still facing this issue

@githubDiversity
Copy link

githubDiversity commented Oct 18, 2022

EDIT: My apologies. I realize now that I was testing in a virtualized environment. Making it the host responsibility to properly reset the PCIe device.
So whether OpenWrt 22.3.1 still has the same issue or not I really can't say

@szarpaj
Copy link

szarpaj commented Feb 2, 2023

So whether OpenWrt 22.3.1 still has the same issue or not I really can't say

I have a TOTOLINK A7000R router flashed with 22.03.2 r19803-9a599fee93. Was working without issues for 100+ days. Today after upgrade to 22.03.3 the 2.4 GHz wireless went down. Downgrade to 22.03.2 - 2.4 GHz still down. Reboots didnt help, had to poweroff the device and then it worked. So it seems it's still here.

@jiho0307
Copy link

I install 22.3.3 on my Iptime A6ns-M (MT7621 SoC, MT7615E). After install, I can only use 5G radio. System log say "hostapd: Could not set interface wlan1 flags (UP): I/O error". After hard reset, it works but reboot on luci make 2.4 GHz radio fails...
It still happened in 22.3.3...

@carlosfilhoadm
Copy link

Thank you @tinxx

The script is working on a Totolink A7000R running 22.03.5

@szarpaj
Copy link

szarpaj commented Aug 11, 2024

Update: I still have a TOTOLINK A7000R. I've had troubles sometimes with wifi, but hard reboot solved them. Today wifi refused to work at all - hard reboot doesnt solve those issues. Update OpenWRT to latest stable - still no. Latest snapthost - still no.
There's the log - would be nice to solve this:

[   11.590274] pci 0000:00:00.0: enabling device (0000 -> 0003)
[   11.601646] mt7615e 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[   11.620083] mt7615e 0000:01:00.0: disabling ASPM L0s L1
[   11.636753] ------------[ cut here ]------------
[   11.645962] WARNING: CPU: 0 PID: 724 at target-mipsel_24kc_musl/linux-ramips_mt7621/mt76-2024.07.13~3b47d9df/mt7615/eeprom.c:31 mt7615_eeprom_init+0x4e4/0x500 [mt7615_common]
[   11.676988] Modules linked in: mt7615e(O+) mt7615_common(O) mt76_connac_lib(O) mt76(O) mac80211(O) cfg80211(O) slhc nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c hwmon crc_ccitt compat(O) sha512_generic sha256_generic libsha256 sha1_generic seqiv sha3_generic drbg md5 crypto_hw_eip93 hmac geniv rng des_generic libdes cmac authencesn authenc leds_gpio gpio_button_hotplug(O) crc32c_generic
[   11.753848] CPU: 0 PID: 724 Comm: kmodloader Tainted: G           O       6.6.44 #0
[   11.769093] Stack : 809ca440 80088a28 000000e7 00000004 00000000 00000000 83d93b6c 80b90000
[   11.785754]         80a50000 808ce648 00000000 00000000 00000000 00000001 83d93b18 81845780
[   11.802409]         00000000 00000000 808ce648 83d93a50 ffffefff 00000000 ffffffea 00000000
[   11.819066]         83d93a5c 000000e8 809d20f0 ffffffff 808ce648 00000001 00000000 83f18394
[   11.835722]         00000009 00000000 809ca440 000f0040 00000018 8048bdfc 00000000 80b90000
[   11.852377]         ...
[   11.857242] Call Trace:
[   11.862094] [<800079f4>] show_stack+0x28/0xf0
[   11.870792] [<807cb2b0>] dump_stack_lvl+0x60/0x80
[   11.880172] [<8002d930>] __warn+0xb0/0xe4
[   11.888173] [<8002da8c>] warn_slowpath_fmt+0x128/0x188
[   11.898406] [<83f18394>] mt7615_eeprom_init+0x4e4/0x500 [mt7615_common]
[   11.911600] [<83efc288>] mt7615_register_device+0xb0/0x230 [mt7615e]
[   11.924276] [<83efe100>] mt7615_mmio_probe+0x184/0x244 [mt7615e]
[   11.936250] [<80441170>] pci_device_probe+0x90/0x114
[   11.936624] urngd: v1.0.2 started.
[   11.946156] [<804a072c>] really_probe+0xd4/0x368
[   11.946192] [<804a0bf4>] driver_probe_device+0x4c/0x154
[   11.946207] [<804a0f08>] __driver_attach+0xb0/0x1c0
[   11.982210] [<8049e318>] bus_for_each_dev+0x70/0xb8
[   11.991925] [<8049f8ac>] bus_add_driver+0xf8/0x224
[   12.001461] [<804a1bf8>] driver_register+0x88/0x154
[   12.011177] [<80001644>] do_one_initcall+0x50/0x2e0
[   12.020884] [<800aba24>] do_init_module+0x5c/0x214
[   12.030433] [<800adc50>] sys_init_module+0x1b0/0x1fc
[   12.040316] [<8001356c>] syscall_common+0x34/0x58
[   12.049691]
[   12.052693] ---[ end trace 0000000000000000 ]---
[   12.065158] mt7615e 0000:01:00.0: registering led 'mt76-phy0'
[   12.078350] ieee80211 phy0: Selected rate control algorithm 'minstrel_ht'
[   12.081147] pci 0000:00:01.0: enabling device (0000 -> 0003)
[   12.084874] mt7615e 0000:01:00.0: Firmware is not ready for download
[   12.092558] mt7615e 0000:02:00.0: Unable to change power state from D3cold to D0, device inaccessible
[   12.123739] mt7615e 0000:02:00.0: disabling ASPM L0s L1
[   12.147379] mt7615e 0000:02:00.0: registering led 'mt76-phy1'
[   12.160694] ieee80211 phy1: Selected rate control algorithm 'minstrel_ht'
[   12.160754] ieee80211 phy1: copying sband (band 1) due to VHT EXT NSS BW flag
[   12.165738] mt7615e 0000:02:00.0: Firmware is not ready for download
[   12.260057] PPP generic driver version 2.4.2
[   12.272155] NET: Registered PF_PPPOX protocol family
[   12.815259] kmodloader: done loading kernel modules from /etc/modules.d/*
[   21.037958] mtk_soc_eth 1e100000.ethernet eth0: Link is Down
[   21.076386] mtk_soc_eth 1e100000.ethernet eth0: configuring for fixed/rgmii link mode
[   21.092827] mtk_soc_eth 1e100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[   21.098538] mt7530-mdio mdio-bus:1f lan1: configuring for phy/gmii link mode
[   21.125432] br-lan: port 1(lan1) entered blocking state
[   21.135926] br-lan: port 1(lan1) entered disabled state
[   21.146456] mt7530-mdio mdio-bus:1f lan1: entered allmulticast mode
[   21.159034] mtk_soc_eth 1e100000.ethernet eth0: entered allmulticast mode
[   21.175469] mt7530-mdio mdio-bus:1f lan1: entered promiscuous mode
[   21.208167] mt7530-mdio mdio-bus:1f lan2: configuring for phy/gmii link mode
[   21.223843] br-lan: port 2(lan2) entered blocking state
[   21.234477] br-lan: port 2(lan2) entered disabled state
[   21.245124] mt7530-mdio mdio-bus:1f lan2: entered allmulticast mode
[   21.259820] mt7530-mdio mdio-bus:1f lan2: entered promiscuous mode
[   21.282949] mt7530-mdio mdio-bus:1f lan3: configuring for phy/gmii link mode
[   21.298796] br-lan: port 3(lan3) entered blocking state
[   21.309364] br-lan: port 3(lan3) entered disabled state
[   21.320027] mt7530-mdio mdio-bus:1f lan3: entered allmulticast mode
[   21.334821] mt7530-mdio mdio-bus:1f lan3: entered promiscuous mode
[   21.358323] mt7530-mdio mdio-bus:1f lan4: configuring for phy/gmii link mode
[   21.373822] br-lan: port 4(lan4) entered blocking state
[   21.384372] br-lan: port 4(lan4) entered disabled state
[   21.394979] mt7530-mdio mdio-bus:1f lan4: entered allmulticast mode
[   21.410009] mt7530-mdio mdio-bus:1f lan4: entered promiscuous mode
[   21.436583] mtk_soc_eth 1e100000.ethernet wan: PHY [mt7530-0:04] driver [MediaTek MT7530 PHY] (irq=POLL)
[   21.455614] mtk_soc_eth 1e100000.ethernet wan: configuring for phy/rgmii link mode
[   24.218784] mt7530-mdio mdio-bus:1f lan1: Link is Up - 1Gbps/Full - flow control rx/tx
[   24.234705] br-lan: port 1(lan1) entered blocking state
[   24.245189] br-lan: port 1(lan1) entered forwarding state
[   24.355786] mt7530-mdio mdio-bus:1f lan2: Link is Up - 1Gbps/Full - flow control off
[   24.371800] br-lan: port 2(lan2) entered blocking state
[   24.382380] br-lan: port 2(lan2) entered forwarding state
[   24.693671] mt7530-mdio mdio-bus:1f lan4: Link is Up - 1Gbps/Full - flow control off
[   24.709280] br-lan: port 4(lan4) entered blocking state
[   24.719742] br-lan: port 4(lan4) entered forwarding state
[   25.192148] mt7530-mdio mdio-bus:1f lan3: Link is Up - 1Gbps/Full - flow control rx/tx
[   25.208138] br-lan: port 3(lan3) entered blocking state
[   25.218593] br-lan: port 3(lan3) entered forwarding state
[   25.648283] mtk_soc_eth 1e100000.ethernet wan: Link is Up - 1Gbps/Full - flow control rx/tx

@szarpaj
Copy link

szarpaj commented Aug 11, 2024

After a rmmod all the mt76* modules and modprobe mt7615e there's this (wifi still does not work):

[ 1613.776315] mt7615e 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 1613.794871] mt7615e 0000:01:00.0: disabling ASPM L0s L1
[ 1613.812094] ------------[ cut here ]------------
[ 1613.821382] WARNING: CPU: 3 PID: 3198 at target-mipsel_24kc_musl/linux-ramips_mt7621/mt76-2024.07.13~3b47d9df/mt7615/eeprom.c:31 mt7615_eeprom_init+0x4e4/0x500 [mt7615_common]
[ 1613.852629] Modules linked in: mt7615e(O+) mt7615_common(O) mt76_connac_lib(O) mt76(O) jitterentropy_rng pppoe ppp_async nft_fib_inet nf_flow_table_inet pppox ppp_generic nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mac80211(O) cfg80211(O) slhc nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c hwmon crc_ccitt compat(O) sha512_generic sha256_generic libsha256 sha1_generic seqiv sha3_generic drbg md5 crypto_hw_eip93 hmac geniv rng des_generic libdes cmac authencesn authenc leds_gpio gpio_button_hotplug(O) crc32c_generic [last unloaded: mt76(O)]
[ 1613.991961] CPU: 3 PID: 3198 Comm: modprobe Tainted: G        W  O       6.6.44 #0
[ 1614.007050] Stack : 809ca440 80088a40 00000148 00000004 00000000 00000000 8472fb6c 80b90000
[ 1614.023711]         80a50000 808ce648 00000000 00000000 00000000 00000001 8472fb18 81888640
[ 1614.040364]         00000000 00000000 808ce648 8472fa50 ffffefff 00000000 ffffffea 00000000
[ 1614.057019]         8472fa5c 00000149 809d20f0 ffffffff 808ce648 00000001 00000000 843b8394
[ 1614.073679]         00000009 00000000 809ca440 000f0040 00000018 8048bdfc 0000000c 80b9000c
[ 1614.090330]         ...
[ 1614.095192] Call Trace:
[ 1614.100045] [<800079f4>] show_stack+0x28/0xf0
[ 1614.108741] [<807cb2b0>] dump_stack_lvl+0x60/0x80
[ 1614.118118] [<8002d930>] __warn+0xb0/0xe4
[ 1614.126113] [<8002da8c>] warn_slowpath_fmt+0x128/0x188
[ 1614.136342] [<843b8394>] mt7615_eeprom_init+0x4e4/0x500 [mt7615_common]
[ 1614.149554] [<83e48288>] mt7615_register_device+0xb0/0x230 [mt7615e]
[ 1614.162248] [<83e4a100>] mt7615_mmio_probe+0x184/0x244 [mt7615e]
[ 1614.174221] [<80441170>] pci_device_probe+0x90/0x114
[ 1614.184122] [<804a072c>] really_probe+0xd4/0x368
[ 1614.193323] [<804a0bf4>] driver_probe_device+0x4c/0x154
[ 1614.203722] [<804a0f08>] __driver_attach+0xb0/0x1c0
[ 1614.213431] [<8049e318>] bus_for_each_dev+0x70/0xb8
[ 1614.223134] [<8049f8ac>] bus_add_driver+0xf8/0x224
[ 1614.232667] [<804a1bf8>] driver_register+0x88/0x154
[ 1614.242375] [<80001644>] do_one_initcall+0x50/0x2e0
[ 1614.252080] [<800aba24>] do_init_module+0x5c/0x214
[ 1614.261628] [<800adc50>] sys_init_module+0x1b0/0x1fc
[ 1614.271506] [<8001356c>] syscall_common+0x34/0x58
[ 1614.280872]
[ 1614.283963] ---[ end trace 0000000000000000 ]---
[ 1614.297494] mt7615e 0000:01:00.0: registering led 'mt76-phy4'
[ 1614.310708] ieee80211 phy4: Selected rate control algorithm 'minstrel_ht'
[ 1614.326813] mt7615e 0000:01:00.0: Firmware is not ready for download
[ 1614.327089] mt7615e 0000:02:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 1614.358250] mt7615e 0000:02:00.0: disabling ASPM L0s L1
[ 1614.379658] mt7615e 0000:02:00.0: registering led 'mt76-phy5'
[ 1614.392912] ieee80211 phy5: Selected rate control algorithm 'minstrel_ht'
[ 1614.406667] ieee80211 phy5: copying sband (band 1) due to VHT EXT NSS BW flag
[ 1614.423537] mt7615e 0000:02:00.0: Firmware is not ready for download

@githubDiversity
Copy link

I am sorry that I can not help out on a tech/coding level. For me I kinda moved on since I guessed the devs never imagined that OpenWRT is also awesome virtualized

@vallsur
Copy link

vallsur commented Oct 13, 2024

So whether OpenWrt 22.3.1 still has the same issue or not I really can't say

I have a TOTOLINK A7000R router flashed with 22.03.2 r19803-9a599fee93. Was working without issues for 100+ days. Today after upgrade to 22.03.3 the 2.4 GHz wireless went down. Downgrade to 22.03.2 - 2.4 GHz still down. Reboots didnt help, had to poweroff the device and then it worked. So it seems it's still here.

I faced just with the same issue in OpenWrt 23.05.4 r24012-d8dd03c46f / LuCI openwrt-23.05 branch git-24.086.45142-09d5a38 with my router: TOTOLINK A7000R. 2.4G was not working, 5G was working fine..... Hard reset also fixed the issue, BUT provided work around "rmmod mt7615e" + "reboot" unfortunately did't help ("rmmod mt7615e" only drops both Wi-Fi and after soft reboot 2.4G still isn't working) :( Therefore script provided by @tinxx also did't work ....

@vallsur
Copy link

vallsur commented Oct 13, 2024

Thank you @tinxx

The script is working on a Totolink A7000R running 22.03.5

I would be very thankful if you give me a clue what I am doing wrong with the provided workaround script ,,,,,,

@carlosfilhoadm
Copy link

carlosfilhoadm commented Oct 21, 2024

Hi @vallsur

The script worked on 22.03 and 23.05

Please try this:

vi /etc/init.d/mt7615e

#!/bin/sh /etc/rc.common
STOP=52
stop() {
 rmmod mt7615e
}

chmod 0755 /etc/init.d/mt7615e

/etc/init.d/mt7615e enable

poweroff

Disconnect the device from power, wait a few seconds, reconnect it

Activate wi-fi, then reboot the device to test if the script is working

@szarpaj
Copy link

szarpaj commented Oct 22, 2024

poweroff

It works (at least here and it seems for @wojciech-z) when I power cycle the device without any scritping magic.

@carlosfilhoadm
Copy link

@szarpaj

The script is useful for other situations, like rebooting a remote device or scheduling daily automated reboots.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests