netflow2ng: zmq format for ntopng incorrect? (openflow support)
I am trying to push some data into ntopng 6.0, but it doesn’t receive any flows, despite receiving zmq messages:
So I went ahead and compared the output of netflow2ng (0.0.5) with the output of nprobe (10.5) and they seem to be very different: netflow2ng output
{
"1": 1421,
"10": 0,
"11": 33923,
"12": "61.250.232.215",
"130": "192.168.0.1",
"14": 0,
"15": "61.250.232.215",
"16": 0,
"17": 0,
"2": 6,
"21": 1706022328,
"22": 1706022318,
"29": 0,
"30": 0,
"32": 0,
"4": 6,
"44": 0,
"45": 0,
"5": 0,
"52": 0,
"54": 0,
"58": 0,
"59": 0,
"6": 27,
"60": 4,
"61": 0,
"7": 22,
"8": "89.0.69.55",
"80": "00:00:00:00:00:00",
"81": "00:00:00:00:00:00",
"88": 0,
"89": 0
}
nprobe output
{
"56": "00:00:00:00:00:00",
"57": "00:00:00:00:00:00",
"10": 0,
"14": 0,
"58": 0,
"8": "192.168.0.1",
"12": "192.168.0.4",
"7": 53,
"11": 51996,
"27": "::",
"28": "::",
"60": 4,
"4": 17,
"35632.118": "5",
"35632.560": 0,
"1": 64,
"2": 1,
"23": 0,
"24": 0,
"22": 1706022259,
"21": 1706022259,
"35632.78": 0,
"35632.79": 0,
"35632.509": 0,
"35632.527": 0,
"130": "192.168.0.1",
"61": 2,
"34": 1,
"42": 18,
"35632.471": "192.168.3.28",
"35632.122": "",
"58051": 1,
"225": "0.0.0.0",
"226": "0.0.0.0",
"227": 0,
"228": 0
}
I am using the @NTOPNG@ template on nprobe, which expands to
%IN_SRC_MAC %OUT_DST_MAC %INPUT_SNMP %OUTPUT_SNMP %SRC_VLAN %IPV4_SRC_ADDR %IPV4_DST_ADDR %L4_SRC_PORT %L4_DST_PORT %IPV6_SRC_ADDR %IPV6_DST_ADDR %IP_PROTOCOL_VERSION %PROTOCOL %L7_PROTO %L7_CONFIDENCE %IN_BYTES %IN_PKTS %OUT_BYTES %OUT_PKTS %FIRST_SWITCHED %LAST_SWITCHED %CLIENT_TCP_FLAGS %SERVER_TCP_FLAGS %L7_PROTO_RISK %L7_RISK_SCORE %EXPORTER_IPV4_ADDRESS %DIRECTION %SAMPLING_INTERVAL %TOTAL_FLOWS_EXP %NPROBE_IPV4_ADDRESS %NPROBE_INSTANCE_NAME %FLOW_SOURCE %POST_NAT_SRC_IPV4_ADDR %POST_NAT_DST_IPV4_ADDR %POST_NAPT_SRC_TRANSPORT_PORT %POST_NAPT_DST_TRANSPORT_PORT
Looking at https://github.com/cloudflare/goflow/tree/master/decoders/netflow, I can see how I could manipulate the JSON output by incorporating those additional fields into https://github.com/synfinatic/netflow2ng/blob/main/cmd/zmq.go. However, I am not sure how those “combined” fields (i.e. “35632.527”) are created / what they mean. Also, I am not sure why for example the MAC addresses use different identifiers on the nprobe string (for example 56 - IPFIX_FIELD_sourceMacAddress, 57 - IPFIX_FIELD_postDestinationMacAddress) compared to the identifiers netflow2ng uses (for example 80 - IPFIX_FIELD_destinationMacAddress, 81 - IPFIX_FIELD_postSourceMacAddress). Same goes for a couple of the other fields - I guess the template used by netflow2ng at the moment is an old version, compatible with previous versions of ntopng!?
So, for now at least, ntopng doesn’t receive any flows from netflow2ng. Could you give any pointers on how to incorporate the changes neccessary to support a more current ntopng version?
About this issue
- Original URL
- State: open
- Created 3 years ago
- Reactions: 1
- Comments: 35 (12 by maintainers)
So going back to the beginning… @level20peon tried to show that nprobe is sending different data than netflow2ng. But this is expected. For example, if you look at ntop_flow.h you’ll see there are four different ways the MAC address can be specified:
And that makes sense since packets have a direction (in vs. out) and you have two addresses (src and dst). So the fact that two different netflow messages use different keys for the MAC addresses makes sense and if you look, both nprobe and netflow2ng are sending valid values.
That said, nothing I see in the json output of netflow2ng looks wrong or invalid. Digging through the ntopng code, I don’t see anything obvious that would cause any issues and my testing has shown it works fine with both ntopng v5 and v6.
If you’d like me to investigate further, frankly I need pcaps of the netflow traffic being sent to netflow2ng and more information about how the netflow traffic being generated. @gitet mentioned using OpnSense which I assume is using softflowd? What version? what is the configuration/CLI arguments?
If it helps, here is how my pfSense box is running softflowd v1.0.0:
Furthermore, you’ll get debugging of ZMQ messages with ntopng at
--verbose=5. you may also want to try not sniffing onwlp4s0and just relying on ZMQ.I haven’t had a chance yet to dig deeper, but looks like https://github.com/ntop/ntopng/blob/dev/src/ZMQParserInterface.cpp has changed significantly from 5.6 to 6.0 which most likely explains what is going on here.
src/ZMQCollectorInterface.cpp
include/ntop_defines.h
As you can see from the inline comment, if the condition is not fulfilled and zmq_message _headers are not having correct size, the message is skipped. Are the messages form the netflow2ng conformant to the defined headers?
from the original log message:
20/May/2024 15:28:22 [ZMQCollectorInterface.cpp:353] WARNING: Unsupported publisher version: is your nProbe sender outdated? [319][24][53][2][1]size -> 319 sizeof(struct zmq_msg_hdr_v1) -> 24
h->version -> 53 ZMQ_MSG_VERSION -> 2 ZMQ_COMPATIBILITY_MSG_VERSION) -> 1
@synfinatic , happy to hear that 😃 Yes, I’m jumping to conclusions - flowd is a collector. I’m not particularly fluent with FreeBSD, but as far as I am able to tell, netflow is generated by a NETGRAPH kernel module (
igc0andigc1being my WAN and LAN interfaces):I’ll generate a pcap for you and post here ASAP.
@gitet , no appologies necessary. We all learned something new today. 😃
Anyways, the only reference for
flowdthat google is giving me is for a “netflow collector”- which receives NetFlow packets; not sending them. There is another package calledpfflowdwhich seems to be written by the same person/people, but I’m unable to find the latest version of either. So no idea what OpnSense is using really.So I’m happy to see about why this is a problem and try to fix it, but someone is going to have to provide a pcap of say 100-200 netflow v9 packets. Just attach it here to this ticket, or if you’re super privacy worried about someone seeing your flow data, just email me… my email address is my github username at gmail.
Hi @gitet … in the documentation it says that we should use zmq:// instead of tcp:// … but both give the same result … I have le flows but all throughput and total bytes are 0 just like you
I have the same problem with ntopng 6.0 (stable) —> no traffic being recorded … all 0
Using ntopng with --interface zmq://x.x.x.x:5556
Would love it if this was corrected 😃