frr_exporter: cannot process output of show bgp vrf all ipv4 unicast summary json: unexpected end of JSON input
Hello,
We upgraded towards version 1.1.2 but we are seeing the following issue below:
msg="collector scrape failed" name=bgp duration_seconds=0.058906413 err="cannot process output of show bgp vrf all ipv4 unicast summary json: unexpected end of JSON input: command output: {\n\"default\":{\n \"routerId\":\"10.11.193.2\",\n \"as\":65237,\n \"vrfId\":0,\n \"vrfName\":\"default\",\n \"tableVersion\":2013,\n \"ribCount\":113,\n \"ribMemory\":22600,\n \"peerCount\":10,\n \"peerMemory\":233760,\n \"peerGroupCount\":4,\n \"peerGroupMemory\":256,\n \"peers\":{\n \"10.11.192.45\":{\n \"hostname\":\"borax\",\n \"remoteAs\":65224,\n \"version\":4,\n \"msgRcvd\":15495697,\n \"msgSent\":16552925,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"04:14:21\",\n \"peerUptimeMsec\":15261000,\n \"peerUptimeEstablishedEpoch\":1661762601,\n \"pfxRcd\":2,\n \"pfxSnt\":10,\n \"state\":\"Established\",\n \"connectionsEstablished\":3,\n \"connectionsDropped\":2,\n \"idType\":\"ipv4\"\n },\n \"10.11.192.53\":{\n \"hostname\":\"bruno\",\n \"remoteAs\":65225,\n \"version\":4,\n \"msgRcvd\":146401702,\n \"msgSent\":128100078,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"14w6d15h\",\n \"peerUptimeMsec\":9043150000,\n \"peerUptimeEstablishedEpoch\":1652734712,\n \"pfxRcd\":5,\n \"pfxSnt\":10,\n \"state\":\"Established\",\n \"connectionsEstablished\":1,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.11.192.61\":{\n \"hostname\":\"julius\",\n \"remoteAs\":65226,\n \"version\":4,\n \"msgRcvd\":168398736,\n \"msgSent\":128074402,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"14w0d00h\",\n \"peerUptimeMsec\":8468008000,\n \"peerUptimeEstablishedEpoch\":1653309854,\n \"pfxRcd\":5,\n \"pfxSnt\":10,\n \"state\":\"Established\",\n \"connectionsEstablished\":2,\n \"connectionsDropped\":1,\n \"idType\":\"ipv4\"\n },\n \"10.11.192.69\":{\n \"remoteAs\":65242,\n \"version\":4,\n \"msgRcvd\":3444661,\n \"msgSent\":3015008,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"14w6d16h\",\n \"peerUptimeMsec\":9043336000,\n \"peerUptimeEstablishedEpoch\":1652734526,\n \"pfxRcd\":1,\n \"pfxSnt\":5,\n \"state\":\"Established\",\n \"connectionsEstablished\":1,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.11.192.5\":{\n \"hostname\":\"mewtwo\",\n \"remoteAs\":65001,\n \"version\":4,\n \"msgRcvd\":3013708,\n \"msgSent\":3013717,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"05:29:23\",\n \"peerUptimeMsec\":19763000,\n \"peerUptimeEstablishedEpoch\":1661758099,\n \"pfxRcd\":5,\n \"pfxSnt\":7,\n \"state\":\"Established\",\n \"connectionsEstablished\":48,\n \"connectionsDropped\":47,\n \"idType\":\"ipv4\"\n },\n \"10.11.192.21\":{\n \"remoteAs\":65506,\n \"version\":4,\n \"msgRcvd\":0,\n \"msgSent\":0,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"never\",\n \"peerUptimeMsec\":0,\n \"pfxRcd\":0,\n \"state\":\"Idle (Admin)\",\n \"connectionsEstablished\":0,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"swp48\":{\n \"hostname\":\"DC-IX-CORESW01\",\n \"remoteAs\":65234,\n \"version\":4,\n \"msgRcvd\":131145939,\n \"msgSent\":128100948,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"01w4d15h\",\n \"peerUptimeMsec\":1007473000,\n \"peerUptimeEstablishedEpoch\":1660770389,\n \"pfxRcd\":49,\n \"pfxSnt\":56,\n \"state\":\"Established\",\n \"connectionsEstablished\":2,\n \"connectionsDropped\":1,\n \"idType\":\"interface\"\n },\n \"swp53\":{\n \"hostname\":\"DC-LCL-CORESW01\",\n \"remoteAs\":65236,\n \"version\":4,\n \"msgRcvd\":155333619,\n \"msgSent\":128104743,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"04w0d03h\",\n \"peerUptimeMsec\":2432273000,\n \"peerUptimeEstablishedEpoch\":1659345589,\n \"pfxRcd\":43,\n \"pfxSnt\":56,\n \"state\":\"Established\",\n \"connectionsEstablished\":3,\n \"connectionsDropped\":2,\n \"idType\":\"interface\"\n },\n \"swp54\":{\n \"hostname\":\"DC-LCL-CORESW01\",\n \"remoteAs\":65236,\n \"version\":4,\n \"msgRcvd\":155333620,\n \"msgSent\":128105060,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"04w0d03h\",\n \"peerUptimeMsec\":2432274000,\n \"peerUptimeEstablishedEpoch\":1659345588,\n \"pfxRcd\":43,\n \"pfxSnt\":56,\n \"state\":\"Established\",\n \"connectionsEstablished\":3,\n \"connectionsDropped\":2,\n \"idType\":\"interface\"\n },\n \"swp56\":{\n \"hostname\":\"DC-IX-CORESW02\",\n \"remoteAs\":65235,\n \"version\":4,\n \"msgRcvd\":17468827,\n \"msgSent\":16416336,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"01w4d16h\",\n \"peerUptimeMsec\":1008055000,\n \"peerUptimeEstablishedEpoch\":1660769807,\n \"pfxRcd\":41,\n \"pfxSnt\":56,\n \"state\":\"Established\",\n \"connectionsEstablished\":1,\n \"connectionsDropped\":0,\n \"idType\":\"interface\"\n }\n },\n \"failedPeers\":1,\n \"totalPeers\":10,\n \"dynamicPeers\":0,\n \"bestPath\":{\n \"multiPathRelax\":\"false\"\n }\n}\n,\n\"nni\":{\n \"routerId\":\"10.11.194.2\",\n \"as\":65267,\n \"vrfId\":194,\n \"vrfName\":\"nni\",\n \"tableVersion\":143706,\n \"ribCount\":541,\n \"ribMemory\":108200,\n \"peerCount\":34,\n \"peerMemory\":794784,\n \"peerGroupCount\":4,\n \"peerGroupMemory\":256,\n \"peers\":{\n \"172.31.255.213\":{\n \"remoteAs\":65100,\n \"version\":4,\n \"msgRcvd\":83516,\n \"msgSent\":80581,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"03w6d23h\",\n \"peerUptimeMsec\":2416363000,\n \"peerUptimeEstablishedEpoch\":1659361499,\n \"pfxRcd\":2,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":1,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.11.192.13\":{\n \"hostname\":\"mewtwo\",\n \"remoteAs\":65002,\n \"version\":4,\n \"msgRcvd\":3029477,\n \"msgSent\":3053105,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"05:29:21\",\n \"peerUptimeMsec\":19761000,\n \"peerUptimeEstablishedEpoch\":1661758101,\n \"pfxRcd\":39,\n \"pfxSnt\":272,\n \"state\":\"Established\",\n \"connectionsEstablished\":254,\n \"connectionsDropped\":253,\n \"idType\":\"ipv4\"\n },\n \"10.11.192.29\":{\n \"remoteAs\":65506,\n \"version\":4,\n \"msgRcvd\":0,\n \"msgSent\":0,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"never\",\n \"peerUptimeMsec\":0,\n \"pfxRcd\":0,\n \"state\":\"Idle (Admin)\",\n \"connectionsEstablished\":0,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.3\":{\n \"remoteAs\":64831,\n \"version\":4,\n \"msgRcvd\":0,\n \"msgSent\":0,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"never\",\n \"peerUptimeMsec\":0,\n \"pfxRcd\":0,\n \"state\":\"Idle (Admin)\",\n \"connectionsEstablished\":0,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.11\":{\n \"remoteAs\":64832,\n \"version\":4,\n \"msgRcvd\":0,\n \"msgSent\":0,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"never\",\n \"peerUptimeMsec\":0,\n \"pfxRcd\":0,\n \"state\":\"Idle (Admin)\",\n \"connectionsEstablished\":0,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.18\":{\n \"remoteAs\":64835,\n \"version\":4,\n \"msgRcvd\":0,\n \"msgSent\":0,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"never\",\n \"peerUptimeMsec\":0,\n \"pfxRcd\":0,\n \"state\":\"Idle (Admin)\",\n \"connectionsEstablished\":0,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.26\":{\n \"remoteAs\":64836,\n \"version\":4,\n \"msgRcvd\":0,\n \"msgSent\":0,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"never\",\n \"peerUptimeMsec\":0,\n \"pfxRcd\":0,\n \"state\":\"Idle (Admin)\",\n \"connectionsEstablished\":0,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.34\":{\n \"remoteAs\":64833,\n \"version\":4,\n \"msgRcvd\":0,\n \"msgSent\":0,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"never\",\n \"peerUptimeMsec\":0,\n \"pfxRcd\":0,\n \"state\":\"Idle (Admin)\",\n \"connectionsEstablished\":0,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.42\":{\n \"remoteAs\":64834,\n \"version\":4,\n \"msgRcvd\":0,\n \"msgSent\":0,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"never\",\n \"peerUptimeMsec\":0,\n \"pfxRcd\":0,\n \"state\":\"Idle (Admin)\",\n \"connectionsEstablished\":0,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.50\":{\n \"remoteAs\":64801,\n \"version\":4,\n \"msgRcvd\":10257749,\n \"msgSent\":8978507,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"17:59:20\",\n \"peerUptimeMsec\":64760000,\n \"peerUptimeEstablishedEpoch\":1661713102,\n \"pfxRcd\":0,\n \"state\":\"Connect\",\n \"connectionsEstablished\":7,\n \"connectionsDropped\":7,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.58\":{\n \"remoteAs\":64803,\n \"version\":4,\n \"msgRcvd\":10330891,\n \"msgSent\":9043125,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"3d15h44m\",\n \"peerUptimeMsec\":315851000,\n \"peerUptimeEstablishedEpoch\":1661462011,\n \"pfxRcd\":9,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":18,\n \"connectionsDropped\":17,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.66\":{\n \"remoteAs\":64805,\n \"version\":4,\n \"msgRcvd\":10197271,\n \"msgSent\":8925883,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"1d21h33m\",\n \"peerUptimeMsec\":163995000,\n \"peerUptimeEstablishedEpoch\":1661613867,\n \"pfxRcd\":1,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":30,\n \"connectionsDropped\":29,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.74\":{\n \"remoteAs\":64807,\n \"version\":4,\n \"msgRcvd\":10332021,\n \"msgSent\":9043276,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"3d15h44m\",\n \"peerUptimeMsec\":315851000,\n \"peerUptimeEstablishedEpoch\":1661462011,\n \"pfxRcd\":7,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":17,\n \"connectionsDropped\":16,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.82\":{\n \"remoteAs\":64809,\n \"version\":4,\n \"msgRcvd\":10329557,\n \"msgSent\":9041493,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"3d06h51m\",\n \"peerUptimeMsec\":283912000,\n \"peerUptimeEstablishedEpoch\":1661493950,\n \"pfxRcd\":4,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":9,\n \"connectionsDropped\":8,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.90\":{\n \"remoteAs\":64811,\n \"version\":4,\n \"msgRcvd\":10320810,\n \"msgSent\":9032954,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"1d21h33m\",\n \"peerUptimeMsec\":163991000,\n \"peerUptimeEstablishedEpoch\":1661613871,\n \"pfxRcd\":1,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":82,\n \"connectionsDropped\":81,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.98\":{\n \"remoteAs\":64812,\n \"version\":4,\n \"msgRcvd\":3441595,\n \"msgSent\":3012163,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"4d03h35m\",\n \"peerUptimeMsec\":358535000,\n \"peerUptimeEstablishedEpoch\":1661419327,\n \"pfxRcd\":3,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":12,\n \"connectionsDropped\":11,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.106\":{\n \"remoteAs\":64813,\n \"version\":4,\n \"msgRcvd\":3443567,\n \"msgSent\":3013639,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"06w4d06h\",\n \"peerUptimeMsec\":3997488000,\n \"peerUptimeEstablishedEpoch\":1657780374,\n \"pfxRcd\":3,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":4,\n \"connectionsDropped\":3,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.114\":{\n \"remoteAs\":64814,\n \"version\":4,\n \"msgRcvd\":10297446,\n \"msgSent\":9013735,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"4d15h28m\",\n \"peerUptimeMsec\":401290000,\n \"peerUptimeEstablishedEpoch\":1661376572,\n \"pfxRcd\":4,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":12,\n \"connectionsDropped\":11,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.122\":{\n \"remoteAs\":64815,\n \"version\":4,\n \"msgRcvd\":10331908,\n \"msgSent\":9043123,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"03w6d14h\",\n \"peerUptimeMsec\":2385921000,\n \"peerUptimeEstablishedEpoch\":1659391941,\n \"pfxRcd\":2,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":5,\n \"connectionsDropped\":4,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.130\":{\n \"remoteAs\":64816,\n \"version\":4,\n \"msgRcvd\":10332045,\n \"msgSent\":9042964,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"02w6d06h\",\n \"peerUptimeMsec\":1749628000,\n \"peerUptimeEstablishedEpoch\":1660028234,\n \"pfxRcd\":3,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":9,\n \"connectionsDropped\":8,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.138\":{\n \"remoteAs\":64817,\n \"version\":4,\n \"msgRcvd\":10330314,\n \"msgSent\":9042215,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"02w0d00h\",\n \"peerUptimeMsec\":1210740000,\n \"peerUptimeEstablishedEpoch\":1660567122,\n \"pfxRcd\":8,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":43,\n \"connectionsDropped\":42,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.146\":{\n \"remoteAs\":64818,\n \"version\":4,\n \"msgRcvd\":3247973,\n \"msgSent\":3014469,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"3d15h44m\",\n \"peerUptimeMsec\":315855000,\n \"peerUptimeEstablishedEpoch\":1661462007,\n \"pfxRcd\":14,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":5,\n \"connectionsDropped\":4,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.154\":{\n \"remoteAs\":64819,\n \"version\":4,\n \"msgRcvd\":0,\n \"msgSent\":0,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"never\",\n \"peerUptimeMsec\":0,\n \"pfxRcd\":0,\n \"state\":\"Idle (Admin)\",\n \"connectionsEstablished\":0,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.162\":{\n \"remoteAs\":64820,\n \"version\":4,\n \"msgRcvd\":0,\n \"msgSent\":0,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"never\",\n \"peerUptimeMsec\":0,\n \"pfxRcd\":0,\n \"state\":\"Idle (Admin)\",\n \"connectionsEstablished\":0,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.170\":{\n \"remoteAs\":64821,\n \"version\":4,\n \"msgRcvd\":0,\n \"msgSent\":0,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"never\",\n \"peerUptimeMsec\":0,\n \"pfxRcd\":0,\n \"state\":\"Idle (Admin)\",\n \"connectionsEstablished\":0,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.178\":{\n \"remoteAs\":64822,\n \"version\":4,\n \"msgRcvd\":3444111,\n \"msgSent\":3014256,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"02w0d00h\",\n \"peerUptimeMsec\":1210740000,\n \"peerUptimeEstablishedEpoch\":1660567122,\n \"pfxRcd\":2,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":39,\n \"connectionsDropped\":38,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.186\":{\n \"remoteAs\":64837,\n \"version\":4,\n \"msgRcvd\":10332039,\n \"msgSent\":9043230,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"02w4d22h\",\n \"peerUptimeMsec\":1636077000,\n \"peerUptimeEstablishedEpoch\":1660141785,\n \"pfxRcd\":2,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":29,\n \"connectionsDropped\":28,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.194\":{\n \"remoteAs\":64802,\n \"version\":4,\n \"msgRcvd\":0,\n \"msgSent\":0,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"never\",\n \"peerUptimeMsec\":0,\n \"pfxRcd\":0,\n \"state\":\"Idle (Admin)\",\n \"connectionsEstablished\":0,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.202\":{\n \"remoteAs\":64804,\n \"version\":4,\n \"msgRcvd\":3390967,\n \"msgSent\":3014463,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"5d00h53m\",\n \"peerUptimeMsec\":435197000,\n \"peerUptimeEstablishedEpoch\":1661342665,\n \"pfxRcd\":2,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":5,\n \"connectionsDropped\":4,\n \"idType\":\"ipv4\"\n },\n \"10.189.100.218\":{\n \"remoteAs\":65041,\n \"version\":4,\n \"msgRcvd\":1973897,\n \"msgSent\":1816990,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"09w0d02h\",\n \"peerUptimeMsec\":5450717000,\n \"peerUptimeEstablishedEpoch\":1656327145,\n \"pfxRcd\":113,\n \"pfxSnt\":1,\n \"state\":\"Established\",\n \"connectionsEstablished\":1,\n \"connectionsDropped\":0,\n \"idType\":\"ipv4\"\n },\n \"swp48.1999\":{\n \"hostname\":\"DC-IX-CORESW01\",\n \"remoteAs\":65264,\n \"version\":4,\n \"msgRcvd\":3038670,\n \"msgSent\":3051910,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"01w4d15h\",\n \"peerUptimeMsec\":1007042000,\n \"peerUptimeEstablishedEpoch\":1660770820,\n \"pfxRcd\":197,\n \"pfxSnt\":273,\n \"state\":\"Established\",\n \"connectionsEstablished\":2,\n \"connectionsDropped\":1,\n \"idType\":\"interface\"\n },\n \"swp53.1999\":{\n \"hostname\":\"DC-LCL-CORESW01\",\n \"remoteAs\":65266,\n \"version\":4,\n \"msgRcvd\":3054316,\n \"msgSent\":3052182,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"04w0d03h\",\n \"peerUptimeMsec\":2432273000,\n \"peerUptimeEstablishedEpoch\":1659345589,\n \"pfxRcd\":51,\n \"pfxSnt\":273,\n \"state\":\"Established\",\n \"connectionsEstablished\":3,\n \"connectionsDropped\":2,\n \"idType\":\"interface\"\n },\n \"swp54.1999\":{\n \"hostname\":\"DC-LCL-CORESW01\",\n \"remoteAs\":65266,\n \"version\":4,\n \"msgRcvd\":3054356,\n \"msgSent\":3052187,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"04w0d03h\",\n \"peerUptimeMsec\":2432274000,\n \"peerUptimeEstablishedEpoch\":1659345588,\n \"pfxRcd\":51,\n \"pfxSnt\":273,\n \"state\":\"Established\",\n \"connectionsEstablished\":3,\n \"connectionsDropped\":2,\n \"idType\":\"interface\"\n },\n \"swp56.1999\":{\n \"hostname\":\"DC-IX-CORESW02\",\n \"remoteAs\":65265,\n \"version\":4,\n \"msgRcvd\":338775,\n \"msgSent\":339646,\n \"tableVersion\":0,\n \"outq\":0,\n \"inq\":0,\n \"peerUptime\":\"01w4d16h\",\n \"peerUptimeMsec\":1008054000,\n \"peerUptimeEstablishedEpoch\":1660769808,\n \"pfxRcd\":219,\n \"pfxSnt\":273,\n \"state\":\"Established\",\n \"connectionsEstablished\":1,\n \"connectionsDropped\":0,\n \"idType\":\"interface\"\n }\n },\n \"failedPeers\":12,\n \"totalPeers\":34,\n \"dynamicPeers\":0,\n \"bestPath\":{\n \"multiPathRelax\":\"false\"\n }\n}\n}\n"
frr_exporter service file:
cat /etc/systemd/system/frr_exporter@.service
[Unit]
Description=Prometheus FRR Exporter
After=network-online.target
[Service]
Type=simple
User=root
Group=root
#ExecStart=/usr/local/bin/frr_exporter
ExecStart=/usr/local/bin/frr_exporter --frr.socket.dir-path="/var/run/frr" --no-collector.ospf --no-collector.bfd --collector.bgp6 --collector.bgpl2vpn --collector.bgp.peer-descriptions --collector.bgp.peer-descriptions.plain-text
SyslogIdentifier=frr_exporter
Restart=always
RestartSec=1
StartLimitInterval=0
[Install]
WantedBy=multi-user.target
FRR Version (cumulus linux 4.4.4)
FRRouting 7.5+cl4.4.0u4 (DC-LCL-CORESW02).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--enable-exampledir=/usr/share/doc/frr/examples/' '--localstatedir=/var/run/frr' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc/frr' '--with-vtysh-pager=/usr/bin/pager' '--libdir=/usr/lib/x86_64-linux-gnu/frr' '--with-moduledir=/usr/lib/x86_64-linux-gnu/frr/modules' '--disable-dependency-tracking' '--disable-dev-build' '--enable-systemd=yes' '--enable-rpki' '--with-libpam' '--enable-doc' '--enable-doc-html' '--enable-snmp' '--enable-fpm' '--disable-zeromq' '--enable-ospfapi' '--disable-bgp-vnc' '--enable-multipath=128' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-configfile-mask=0640' '--enable-logfile-mask=0640' '--disable-address-sanitizer' '--enable-protobuf' '--enable-cumulus=yes' '--enable-csmgr=yes' '--enable-datacenter=yes' '--enable-bfdd=no' '--enable-sharpd=yes' '--enable-grpc=yes' 'build_alias=x86_64-linux-gnu' 'PYTHON=python3'
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 18 (11 by maintainers)
Thanks for granting me access to troubleshoot, as it helped me determine the cause.
FRR was prematurely returning data read from the FRR socket due to this line. The point of return in the text output was different across each run, which is odd, but I imagine FRR is returning
\x00to split up large data.Luckily, no code changes are needed as @dswarbrick refactored how FRR Exporter interfaces with the FRR socket (https://github.com/tynany/frr_exporter/pull/81), fixing the issue.
The issue is no longer present in v1.1.3.
@tynany It probably wouldn’t hurt to sprinkle some debug logging of such command outputs, which can be left in there permanently and optionally viewed by end users, depending on their configured log level. This can really help when trying to troubleshoot issues such as this, where we don’t entirely trust the output of some external source that we need to parse.
I would also recommend wrapping errors with some context, rather than just returning them raw. For example, this kind of thing would help identify where errors have sprung up from, especially if they’re buried multiple functions deep, e.g. in
processBGPPeerDesc:This will preserve (wrap) the original error thrown by json.Unmarshal, but provides some user-friendly context as well.