start-os: [bug]: .local access sometimes breaks on all services.
Not sure how to reproduce. I was testing spark and the .local address worked briefly, then stopped. I went and tried to visit other services but they don’t work either. I tried on both mac and linux and that also doesn’t work. All I get is this, on all services:
Nothing that suspicious in the logs, though maybe I missed something:
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.087 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Loading services for mDNS
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.091 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "embassy-30297ba3.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.092 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for pjzz3s232yrelgbrcjt2im74hnfe3t7crjlw55smrqmxojp5tbiehoad.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.095 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "pjzz3s232yrelgbrcjt2im74hnfe3t7crjlw55smrqmxojp5tbiehoad.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.096 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for r7l7qgzlp3o7iciz53upx4t2hu7l7hj4fpuqiqksxhvirosodhlcg5id.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.098 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "r7l7qgzlp3o7iciz53upx4t2hu7l7hj4fpuqiqksxhvirosodhlcg5id.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.100 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for skgq5mnyd6sxchfpglxw2qre7dlauxfujnkrvplzuqd5rbv5y7bxfbid.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.103 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "skgq5mnyd6sxchfpglxw2qre7dlauxfujnkrvplzuqd5rbv5y7bxfbid.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.104 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for 5bfhafo4jze47wkxbbxsm5s6hilfjqcbd34wuteeds4mqc6tq4ftqmid.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.106 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "5bfhafo4jze47wkxbbxsm5s6hilfjqcbd34wuteeds4mqc6tq4ftqmid.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.107 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for tryniygn3ireoqx2umxypwnfc4gtswhgmgtulcdvjdu6nftkba7xqhqd.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.110 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "tryniygn3ireoqx2umxypwnfc4gtswhgmgtulcdvjdu6nftkba7xqhqd.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.110 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for 76lx5hfsfaxhkrztapw52grgol62qra73bd57yhc6jvyzfzsio2do5id.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.112 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "76lx5hfsfaxhkrztapw52grgol62qra73bd57yhc6jvyzfzsio2do5id.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.112 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for 7tnq3yaeci6lrw7v2ova74kl3ttcrqks2j26rdo5il46owzj5lxvkiyd.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.114 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "7tnq3yaeci6lrw7v2ova74kl3ttcrqks2j26rdo5il46owzj5lxvkiyd.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.114 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for z57zpgc6yzzkbl2yqcchoao5gmk4frcsk65xqm2acwzg4owz73njqpyd.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.116 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "z57zpgc6yzzkbl2yqcchoao5gmk4frcsk65xqm2acwzg4owz73njqpyd.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.116 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for yliha5qbq54onrpnmzylsgvgoacnhbwxxadqazlno43bw7mzz3z6qcyd.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.117 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "yliha5qbq54onrpnmzylsgvgoacnhbwxxadqazlno43bw7mzz3z6qcyd.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.118 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for q6z5xc5zc2m4gvv4zcrk6cyipw3uzc4jlake6bwst44ap3h3oaqhz6id.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.119 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "q6z5xc5zc2m4gvv4zcrk6cyipw3uzc4jlake6bwst44ap3h3oaqhz6id.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.119 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for 77cmjrq5qioeyplo5njaiu44zorv5xbgjaumkzdo54qn4m2r3ihcs6yd.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.129 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "77cmjrq5qioeyplo5njaiu44zorv5xbgjaumkzdo54qn4m2r3ihcs6yd.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.130 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for ghymfs422rvwqmoi3wgv4qddjlvhw5vgr65zkshimdpdobq3kfwcozid.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.132 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "ghymfs422rvwqmoi3wgv4qddjlvhw5vgr65zkshimdpdobq3kfwcozid.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.132 DEBUG run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Adding mdns CNAME entry for gfoim2ptc2h7y24yphg4vifoiadiosrp7iv6y36n6362g7dtdwkhqkad.local
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.135 INFO run_main:remove{pkg_id=PackageId(Id("spark-wallet"))}: embassy::net::mdns: Published "gfoim2ptc2h7y24yphg4vifoiadiosrp7iv6y36n6362g7dtdwkhqkad.local"
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.684 ERROR embassy::manager: service crashed: 1: uncaughtException, stopping process
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Error: connect ENOENT /mnt/c-lightning/shared/lightning-rpc
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1157:16)
Jan 06 17:30:51 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:51.684 INFO run_main: embassy::net: Generating SSL Certificate mountpoints for spark-wallet
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.769 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Loading services for mDNS
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.771 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "embassy-30297ba3.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.772 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for pjzz3s232yrelgbrcjt2im74hnfe3t7crjlw55smrqmxojp5tbiehoad.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.774 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "pjzz3s232yrelgbrcjt2im74hnfe3t7crjlw55smrqmxojp5tbiehoad.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.774 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for r7l7qgzlp3o7iciz53upx4t2hu7l7hj4fpuqiqksxhvirosodhlcg5id.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.778 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "r7l7qgzlp3o7iciz53upx4t2hu7l7hj4fpuqiqksxhvirosodhlcg5id.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.778 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for skgq5mnyd6sxchfpglxw2qre7dlauxfujnkrvplzuqd5rbv5y7bxfbid.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.780 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "skgq5mnyd6sxchfpglxw2qre7dlauxfujnkrvplzuqd5rbv5y7bxfbid.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.780 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for 5bfhafo4jze47wkxbbxsm5s6hilfjqcbd34wuteeds4mqc6tq4ftqmid.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.782 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "5bfhafo4jze47wkxbbxsm5s6hilfjqcbd34wuteeds4mqc6tq4ftqmid.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.783 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for tryniygn3ireoqx2umxypwnfc4gtswhgmgtulcdvjdu6nftkba7xqhqd.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.785 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "tryniygn3ireoqx2umxypwnfc4gtswhgmgtulcdvjdu6nftkba7xqhqd.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.785 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for 76lx5hfsfaxhkrztapw52grgol62qra73bd57yhc6jvyzfzsio2do5id.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.791 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "76lx5hfsfaxhkrztapw52grgol62qra73bd57yhc6jvyzfzsio2do5id.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.792 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for 7tnq3yaeci6lrw7v2ova74kl3ttcrqks2j26rdo5il46owzj5lxvkiyd.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.794 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "7tnq3yaeci6lrw7v2ova74kl3ttcrqks2j26rdo5il46owzj5lxvkiyd.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.795 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for z57zpgc6yzzkbl2yqcchoao5gmk4frcsk65xqm2acwzg4owz73njqpyd.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.797 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "z57zpgc6yzzkbl2yqcchoao5gmk4frcsk65xqm2acwzg4owz73njqpyd.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.797 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for yliha5qbq54onrpnmzylsgvgoacnhbwxxadqazlno43bw7mzz3z6qcyd.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.800 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "yliha5qbq54onrpnmzylsgvgoacnhbwxxadqazlno43bw7mzz3z6qcyd.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.801 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for q6z5xc5zc2m4gvv4zcrk6cyipw3uzc4jlake6bwst44ap3h3oaqhz6id.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.805 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "q6z5xc5zc2m4gvv4zcrk6cyipw3uzc4jlake6bwst44ap3h3oaqhz6id.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.805 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for 77cmjrq5qioeyplo5njaiu44zorv5xbgjaumkzdo54qn4m2r3ihcs6yd.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.807 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "77cmjrq5qioeyplo5njaiu44zorv5xbgjaumkzdo54qn4m2r3ihcs6yd.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.807 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for ghymfs422rvwqmoi3wgv4qddjlvhw5vgr65zkshimdpdobq3kfwcozid.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.809 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "ghymfs422rvwqmoi3wgv4qddjlvhw5vgr65zkshimdpdobq3kfwcozid.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.809 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for tbsrlmmrfjgcuca2gqrnjdcvrin775hxubccupq4c56qiae4eh4vyhqd.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.814 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "tbsrlmmrfjgcuca2gqrnjdcvrin775hxubccupq4c56qiae4eh4vyhqd.local"
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.814 DEBUG run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Adding mdns CNAME entry for gfoim2ptc2h7y24yphg4vifoiadiosrp7iv6y36n6362g7dtdwkhqkad.local
Jan 06 17:30:53 embassy-30297ba3 embassyd[4493]: Jan 06 17:30:53.817 INFO run_main:add{pkg_id=PackageId(Id("spark-wallet")) ip=172.18.0.7}: embassy::net::mdns: Published "gfoim2ptc2h7y24yphg4vifoiadiosrp7iv6y36n6362g7dtdwkhqkad.local"
Jan 06 17:34:08 embassy-30297ba3 embassyd[4493]: Jan 06 17:34:08.595 DEBUG embassy::net::tor: Attempting to self-check tor address
Jan 06 17:34:09 embassy-30297ba3 embassyd[4493]: Jan 06 17:34:09.934 DEBUG embassy::net::tor: Successfully verified main tor address liveness at na63or7ygqykbthyuoe2lsxihqmspbzciwn5vhnsyo3xmkfgjjuy56yd.onion
Jan 06 17:39:09 embassy-30297ba3 embassyd[4493]: Jan 06 17:39:09.936 DEBUG embassy::net::tor: Attempting to self-check tor address
Jan 06 17:39:15 embassy-30297ba3 embassyd[4493]: Jan 06 17:39:15.091 DEBUG embassy::net::tor: Successfully verified main tor address liveness at na63or7ygqykbthyuoe2lsxihqmspbzciwn5vhnsyo3xmkfgjjuy56yd.onion
Note: system UI works on .local, just not the services.
Also, .onion access still works.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 22 (22 by maintainers)
Attempted fix in #1108 is gonna be DOA. New approach we are going to take is refactoring the net controller to have install, uninstall, start and stop methods rather than a simple add/remove dynamic. This should reduce the thrash in mdns and should keep it stable in the presence of service crash loops.
I was able to reproduce the same on latest BTCPayServer from Alpha (witch fails in the crash loop). All services are not accessible over .local (.onion addresses work just fine) Your embassy-q1w2e3.local address is accessible as well.
This is just a sample peace of
embassydlog (is flooded massively)I’ll start doing a battery of UAT today and see if it comes up then. That said, leaving this as a P1 when we can’t reliably reproduce it may not be tenable. I’ll punt on attending to this immediately and keep a passive eye out.