From jake at elsif.net Wed May 13 13:58:37 2009 From: jake at elsif.net (elsif) Date: Wed, 13 May 2009 09:58:37 -0400 (EDT) Subject: [dsc] "No data to display at this time". Message-ID: <20090513094642.T60222@disintegration.igs.net> Using dsc-200901261740 on FreeBSD. The collector works. Transferring using rsync to the presenter works. refile-and-grok.sh works. This leaves me with: /var/dsc/data/cira/yow.ca-servers.ca/20090513/*.dat, files that are being populated every minute with valid data. In configuring the presenter, I did: server cira yow.ca-servers.ca domain_list cira ca valid_domains yow.ca-servers.ca cira dsc-grapher.pl loads via browser (Firefox), shows 'cira' as a branch, with 'yow.ca-servers.ca' as a leaf off of that. Using that server/node, and clicking on the links below for the various plot types results in nothing more than 'No data to display at this time'. Any indication here as to what I've done wrong? Thanks all, -Jake From jake at elsif.net Wed May 13 14:08:40 2009 From: jake at elsif.net (elsif) Date: Wed, 13 May 2009 10:08:40 -0400 (EDT) Subject: [dsc] "No data to display at this time". In-Reply-To: <20090513094642.T60222@disintegration.igs.net> References: <20090513094642.T60222@disintegration.igs.net> Message-ID: <20090513100647.F60361@disintegration.igs.net> I should also mention that: dsc-xml-extractor.out has no errors, PERL or otherwise. My entire data dir, /var/dsc/data/* is chgrp'd to my Apache user, and is g+r. My 'cache' dir, and my 'log' dir, are both also chgrp'd to my Apache user, and g+rw. Thx, -Jake On Wed, 13 May 2009, elsif wrote: > Using dsc-200901261740 on FreeBSD. > > The collector works. > Transferring using rsync to the presenter works. > refile-and-grok.sh works. > > This leaves me with: /var/dsc/data/cira/yow.ca-servers.ca/20090513/*.dat, > files that are being populated every minute with valid data. > > In configuring the presenter, I did: > > server cira yow.ca-servers.ca > domain_list cira ca > valid_domains yow.ca-servers.ca cira > > dsc-grapher.pl loads via browser (Firefox), shows 'cira' as a branch, with > 'yow.ca-servers.ca' as a leaf off of that. > > Using that server/node, and clicking on the links below for the various plot > types results in nothing more than 'No data to display at this time'. > > Any indication here as to what I've done wrong? > > Thanks all, > > -Jake > From jake at elsif.net Wed May 13 14:47:21 2009 From: jake at elsif.net (elsif) Date: Wed, 13 May 2009 10:47:21 -0400 (EDT) Subject: [dsc] "No data to display at this time". In-Reply-To: <20090513100647.F60361@disintegration.igs.net> References: <20090513094642.T60222@disintegration.igs.net> <20090513100647.F60361@disintegration.igs.net> Message-ID: <20090513104643.P60666@disintegration.igs.net> Nevermind. I somehow missed the fact that I needed to symlink /usr/local/dsc/data to /var/dsc/data when I'd decided to put the data elsewhere. On Wed, 13 May 2009, elsif wrote: > I should also mention that: > > dsc-xml-extractor.out has no errors, PERL or otherwise. > > My entire data dir, /var/dsc/data/* is chgrp'd to my Apache user, and is g+r. > > My 'cache' dir, and my 'log' dir, are both also chgrp'd to my Apache user, > and g+rw. > > Thx, > > -Jake > > On Wed, 13 May 2009, elsif wrote: > >> Using dsc-200901261740 on FreeBSD. >> >> The collector works. >> Transferring using rsync to the presenter works. >> refile-and-grok.sh works. >> >> This leaves me with: /var/dsc/data/cira/yow.ca-servers.ca/20090513/*.dat, >> files that are being populated every minute with valid data. >> >> In configuring the presenter, I did: >> >> server cira yow.ca-servers.ca >> domain_list cira ca >> valid_domains yow.ca-servers.ca cira >> >> dsc-grapher.pl loads via browser (Firefox), shows 'cira' as a branch, with >> 'yow.ca-servers.ca' as a leaf off of that. >> >> Using that server/node, and clicking on the links below for the various >> plot types results in nothing more than 'No data to display at this time'. >> >> Any indication here as to what I've done wrong? >> >> Thanks all, >> >> -Jake >> > From andrew.ruthven at catalyst.net.nz Tue May 19 00:49:18 2009 From: andrew.ruthven at catalyst.net.nz (Andrew Ruthven) Date: Tue, 19 May 2009 12:49:18 +1200 Subject: [dsc] qtype dataset is empty Message-ID: <1242694158.9064.11.camel@dirk.catalyst.net.nz> Hi, I've just upgraded the collector on one of my monitoring servers to 200901261740 and now all the qtype datasets are empty. All the other datasets appear to be collecting the correct data. The only change I've made to the config file is to add the new datasets that have been introduced: dataset client_port_range dns All:null PortRange:dns_sport_range queries-only; dataset second_ld_vs_rcode dns Rcode:rcode SecondLD:second_ld replies-only max-cells=50; dataset third_ld_vs_rcode dns Rcode:rcode ThirdLD:third_ld replies-only max-cells=50; Any suggestions on what might have gone wrong? Cheers! -- Andrew Ruthven, Wellington, New Zealand At work: andrew.ruthven at catalyst.net.nz At home: andrew at etc.gen.nz GPG fpr: 34CA 12A3 C6F8 B156 72C2 D0D7 D286 CE0C 0C62 B791 From wessels at measurement-factory.com Tue May 19 16:57:33 2009 From: wessels at measurement-factory.com (Duane Wessels) Date: Tue, 19 May 2009 10:57:33 -0600 (MDT) Subject: [dsc] qtype dataset is empty In-Reply-To: <1242694158.9064.11.camel@dirk.catalyst.net.nz> References: <1242694158.9064.11.camel@dirk.catalyst.net.nz> Message-ID: On Tue, 19 May 2009, Andrew Ruthven wrote: > Hi, > > I've just upgraded the collector on one of my monitoring servers to > 200901261740 and now all the qtype datasets are empty. All the other > datasets appear to be collecting the correct data. Hi Andrew, First step is to figure out if the problem is with the collector or with the presenter. Can you look on the presenter and see if the qtype data files are empty or not? ie, $ less /usr/local/dsc/data/$SERVER/$NODE/20090519/qtype.dat DW From andrew.ruthven at catalyst.net.nz Tue May 19 21:04:17 2009 From: andrew.ruthven at catalyst.net.nz (Andrew Ruthven) Date: Wed, 20 May 2009 09:04:17 +1200 Subject: [dsc] qtype dataset is empty In-Reply-To: References: <1242694158.9064.11.camel@dirk.catalyst.net.nz> Message-ID: <1242767057.17734.5.camel@dirk.catalyst.net.nz> Hi Duane, On Tue, 2009-05-19 at 10:57 -0600, Duane Wessels wrote: > > > On Tue, 19 May 2009, Andrew Ruthven wrote: > > > Hi, > > > > I've just upgraded the collector on one of my monitoring servers to > > 200901261740 and now all the qtype datasets are empty. All the other > > datasets appear to be collecting the correct data. > > First step is to figure out if the problem is with the collector or > with the presenter. > > Can you look on the presenter and see if the qtype data files are empty or not? > ie, > > $ less /usr/local/dsc/data/$SERVER/$NODE/20090519/qtype.dat Sure. They contain only timestamps: ... 1242765540 1242765600 1242765660 1242765720 1242765780 1242765840 #MD5 b819dff5df98dc6f500ac911d54a4dec And the XML file from the collector has: ... ... I've attached the collector configuration to this email. What next? Cheers! -- Andrew Ruthven, Wellington, New Zealand At work: andrew.ruthven at catalyst.net.nz At home: andrew at etc.gen.nz GPG fpr: 34CA 12A3 C6F8 B156 72C2 D0D7 D286 CE0C 0C62 B791 -------------- next part -------------- # local_address # # specifies a local IPv4 address. used to determine the # "direction" of an IP packet: sending or receiving or other # local_address 202.46.190.130; # run_dir # # dsc passes this directory to chdir() after starting. # run_dir "/var/spool/dsc/ns1"; # pid_file # # filename where DSC should store its process-id # pid_file "/var/run/dsc-ns1.pid"; # bpf_program # # a berkely packet filter program. it can be used to limit # the number and type of queries that the application receives # from the kernel. note if you limit it to "udp port 53" the # IP-based collectors do not work # # NOTE: bpf_program must GO BEFORE interface # # use this to see only DNS messages #bpf_program "udp port 53"; #bpf_program "udp port 53 or tcp port 53"; #bpf_program "(vlan or not vlan) and port 53 and host 202.46.190.130"; bpf_program "(src port 53 and src host 202.46.190.130 and not vlan) or (vlan 1 and dst port 53 and dst host 202.46.190.130)"; # # use this to see only DNS *queries* #bpf_program "udp dst port 53 and udp[10:2] & 0x8000 = 0"; # interface # # specifies a network interface to sniff packets from. # can specify more than one. # #interface eth0; interface eth1; #interface eth3; # qname_filter # # Defines a custom QNAME-based filter for DNS messages. If # you refer to this named filter on a dataset line, then only # queries or replies for matching QNAMEs will be counted. # The QNAME argument is a regular expression. For example: # # qname_filter WWW-Only ^www\. ; # dataset qtype dns All:null Qtype:qtype queries-only,WWW-Only ; # # datasets # # please see the DSC manual for more information. dataset qtype dns All:null Qtype:qtype queries-only; dataset rcode dns All:null Rcode:rcode replies-only; dataset opcode dns All:null Opcode:opcode queries-only; dataset rcode_vs_replylen dns Rcode:rcode ReplyLen:msglen replies-only; dataset client_subnet dns All:null ClientSubnet:cip4_net queries-only max-cells=200; dataset qtype_vs_qnamelen dns Qtype:qtype QnameLen:qnamelen queries-only; dataset qtype_vs_tld dns Qtype:qtype TLD:tld queries-only,popular-qtypes max-cells=200; dataset certain_qnames_vs_qtype dns CertainQnames:certain_qnames Qtype:qtype queries-only; dataset client_subnet2 dns Class:query_classification ClientSubnet:cip4_net queries-only max-cells=200; dataset client_addr_vs_rcode dns Rcode:rcode ClientAddr:client replies-only max-cells=50; dataset chaos_types_and_names dns Qtype:qtype Qname:qname chaos-class,queries-only; dataset idn_qname dns All:null IDNQname:idn_qname queries-only; dataset edns_version dns All:null EDNSVersion:edns_version queries-only; #dataset edns_bufsiz dns All:null EDNSBufSiz:edns_bufsiz queries-only; dataset do_bit dns All:null D0:do_bit queries-only; dataset rd_bit dns All:null RD:rd_bit queries-only; dataset idn_vs_tld dns All:null TLD:tld queries-only,idn-only; dataset ipv6_rsn_abusers dns All:null ClientAddr:client queries-only,aaaa-or-a6-only,root-servers-net-only max-cells=50; #dataset transport_vs_qtype dns Transport:transport Qtype:qtype queries-only; #dataset domain_vs_qtype dns Qtype:qtype Domain:domain queries-only max-components=2; dataset client_port_range dns All:null PortRange:dns_sport_range queries-only; dataset second_ld_vs_rcode dns Rcode:rcode SecondLD:second_ld replies-only max-cells=50; dataset third_ld_vs_rcode dns Rcode:rcode ThirdLD:third_ld replies-only max-cells=50; dataset direction_vs_ipproto ip Direction:ip_direction IPProto:ip_proto any; # bpf_vlan_tag_byte_order # # Set this to 'host' on FreeBSD-4 where the VLAN id that we # get from BPF appears to already be in host byte order. #bpf_vlan_tag_byte_order host; # match_vlan # # A whitespace-separated list of VLAN IDs. If set, only the # packets with these VLAN IDs will be analyzed by DSC. # #match_vlan 100 200; From wessels at measurement-factory.com Tue May 19 22:11:59 2009 From: wessels at measurement-factory.com (Duane Wessels) Date: Tue, 19 May 2009 16:11:59 -0600 (MDT) Subject: [dsc] qtype dataset is empty In-Reply-To: <1242767057.17734.5.camel@dirk.catalyst.net.nz> References: <1242694158.9064.11.camel@dirk.catalyst.net.nz> <1242767057.17734.5.camel@dirk.catalyst.net.nz> Message-ID: On Wed, 20 May 2009, Andrew Ruthven wrote: > Sure. They contain only timestamps: very puzzling. Can you make a dsc.conf for testing with all the datasets removed except for : dataset qtype dns All:null Qtype:qtype queries-only; Then run it in debug mode: # /usr/local/dsc/bin/dsc -f -d /tmp/dsc-debug.conf Also tell me what operating system you have there and maybe I have something close here and can try to reproduce it. DW From andrew.ruthven at catalyst.net.nz Tue May 19 23:36:26 2009 From: andrew.ruthven at catalyst.net.nz (Andrew Ruthven) Date: Tue, 19 May 2009 23:36:26 +0000 Subject: [dsc] qtype dataset is empty In-Reply-To: References: <1242694158.9064.11.camel@dirk.catalyst.net.nz> <1242767057.17734.5.camel@dirk.catalyst.net.nz> Message-ID: <1242776186.17734.85.camel@dirk.catalyst.net.nz> On Tue, 2009-05-19 at 16:11 -0600, Duane Wessels wrote: > very puzzling. > > Can you make a dsc.conf for testing with all the datasets > removed except for : > > dataset qtype dns All:null Qtype:qtype queries-only; > > Then run it in debug mode: > > # /usr/local/dsc/bin/dsc -f -d /tmp/dsc-debug.conf Sure: srsplog1:/tmp# dsc -f -d ns1-debug.conf adding local address 202.46.190.130 setting current directory to /tmp/ns1 PID file is: /tmp/dsc-ns1.pid BPF program is: (src port 53 and src host 202.46.190.130 and not vlan) or (vlan 1 and dst port 53 and dst host 202.46.190.130) Opening interface eth1 Pcap_init: FD_SET 4 creating dataset qtype writing PID to /tmp/dsc-ns1.pid Running writing to 1242771780.dscdata.xml.XXXQuAam7 renaming to 1242771780.dscdata.xml srsplog1:/tmp# cat ns1/1242771780.dscdata.xml srsplog1:/tmp# If I set the bpf_program to be only "udp port 53 or tcp port 53" I do get some records, but this port is sniffing 3 different nameservers, I need to be able to limit the traffic to only 202.46.190.130. > Also tell me what operating system you have there and maybe > I have something close here and can try to reproduce it. This is Debian Etch on AMD64. The source of DSC (with Debian packaging) I'm using is at: git clone http://git.catalyst.net.nz/dsc.git gitweb http://git.catalyst.net.nz/gw?p=dsc.git Cheers! -- Andrew Ruthven, Wellington, New Zealand At work: andrew.ruthven at catalyst.net.nz At home: andrew at etc.gen.nz GPG fpr: 34CA 12A3 C6F8 B156 72C2 D0D7 D286 CE0C 0C62 B791 From wessels at measurement-factory.com Wed May 20 16:24:59 2009 From: wessels at measurement-factory.com (Duane Wessels) Date: Wed, 20 May 2009 10:24:59 -0600 (MDT) Subject: [dsc] qtype dataset is empty In-Reply-To: <1242776186.17734.85.camel@dirk.catalyst.net.nz> References: <1242694158.9064.11.camel@dirk.catalyst.net.nz> <1242767057.17734.5.camel@dirk.catalyst.net.nz> <1242776186.17734.85.camel@dirk.catalyst.net.nz> Message-ID: On Tue, 19 May 2009, Andrew Ruthven wrote: > git clone http://git.catalyst.net.nz/dsc.git > > gitweb http://git.catalyst.net.nz/gw?p=dsc.git Hi Andrew, Can you apply the attached patch to collector/dsc/qtype_index.c (which just adds some debugging) and then run dsc in debug mode as before? DW -------------- next part -------------- --- ../orig/dsc/collector/dsc/qtype_index.c 2009-05-19 01:11:08.000000000 +0000 +++ collector/dsc/qtype_index.c 2009-05-20 16:19:55.000000000 +0000 @@ -13,14 +13,17 @@ { const dns_message *m = vp; int i; + fprintf(stderr, "qtype_indexer: malformed=%d, qtype=%d\n", m->malformed, m->qtype); if (m->malformed) return -1; for (i = 0; i < next_idx; i++) { if (m->qtype == idx_to_qtype[i]) { + fprintf(stderr, "qtype_indexer: return %d\n", i); return i; } } idx_to_qtype[next_idx] = m->qtype; + fprintf(stderr, "qtype_indexer: return %d\n", next_idx); return next_idx++; } From andrew.ruthven at catalyst.net.nz Wed May 20 22:38:54 2009 From: andrew.ruthven at catalyst.net.nz (Andrew Ruthven) Date: Thu, 21 May 2009 10:38:54 +1200 Subject: [dsc] qtype dataset is empty In-Reply-To: References: <1242694158.9064.11.camel@dirk.catalyst.net.nz> <1242767057.17734.5.camel@dirk.catalyst.net.nz> <1242776186.17734.85.camel@dirk.catalyst.net.nz> Message-ID: <1242859134.13149.45.camel@dirk.catalyst.net.nz> On Wed, 2009-05-20 at 10:24 -0600, Duane Wessels wrote: > Can you apply the attached patch to collector/dsc/qtype_index.c > (which just adds some debugging) and then run dsc in debug mode as > before? Hi Duane, I've applied it, but it didn't produce any output. However, I played around with tcpdump and it seems that the required bpf filter has changed. Using "vlan 1" no longer caught in-bound packets (even though they're in vlan 1). I removed that and now I'm seeing Qtypes again! Thank you for your help! -- Andrew Ruthven, Wellington, New Zealand At work: andrew.ruthven at catalyst.net.nz At home: andrew at etc.gen.nz GPG fpr: 34CA 12A3 C6F8 B156 72C2 D0D7 D286 CE0C 0C62 B791 From wessels at measurement-factory.com Wed May 20 22:50:19 2009 From: wessels at measurement-factory.com (Duane Wessels) Date: Wed, 20 May 2009 16:50:19 -0600 (MDT) Subject: [dsc] qtype dataset is empty In-Reply-To: <1242859134.13149.45.camel@dirk.catalyst.net.nz> References: <1242694158.9064.11.camel@dirk.catalyst.net.nz> <1242767057.17734.5.camel@dirk.catalyst.net.nz> <1242776186.17734.85.camel@dirk.catalyst.net.nz> <1242859134.13149.45.camel@dirk.catalyst.net.nz> Message-ID: On Thu, 21 May 2009, Andrew Ruthven wrote: > I've applied it, but it didn't produce any output. However, I played > around with tcpdump and it seems that the required bpf filter has > changed. Using "vlan 1" no longer caught in-bound packets (even though > they're in vlan 1). I removed that and now I'm seeing Qtypes again! I've experience similar strangeness with VLANs and BPF as well. But originally you said that you were getting data in other datasets and that only qtype data was empty. Was that really the case?? The VLAN checks are applied before any DNS message inspection. I guess I could believe that you were seeing some IP-layer datasets but if the filter was choking on VLANs then all of the DNS datasets should have been empty I think. DW From andrew.ruthven at catalyst.net.nz Wed May 20 22:59:46 2009 From: andrew.ruthven at catalyst.net.nz (Andrew Ruthven) Date: Thu, 21 May 2009 10:59:46 +1200 Subject: [dsc] qtype dataset is empty In-Reply-To: References: <1242694158.9064.11.camel@dirk.catalyst.net.nz> <1242767057.17734.5.camel@dirk.catalyst.net.nz> <1242776186.17734.85.camel@dirk.catalyst.net.nz> <1242859134.13149.45.camel@dirk.catalyst.net.nz> Message-ID: <1242860386.13149.48.camel@dirk.catalyst.net.nz> On Wed, 2009-05-20 at 16:50 -0600, Duane Wessels wrote: > On Thu, 21 May 2009, Andrew Ruthven wrote: > > > I've applied it, but it didn't produce any output. However, I played > > around with tcpdump and it seems that the required bpf filter has > > changed. Using "vlan 1" no longer caught in-bound packets (even though > > they're in vlan 1). I removed that and now I'm seeing Qtypes again! > > I've experience similar strangeness with VLANs and BPF as well. > > But originally you said that you were getting data in other datasets > and that only qtype data was empty. Was that really the case?? The vlan is only set for the queries, not for the replies (go figure). It turns out that the other datasets I'd checked were only looking at the replies. I hadn't checked other datasets that looked at queries (except for all the other Qtype datasets). But looking back through the presenter the other datasets that inspected queries are empty as well. > The VLAN checks are applied before any DNS message inspection. I guess > I could believe that you were seeing some IP-layer datasets but if > the filter was choking on VLANs then all of the DNS datasets should > have been empty I think. Agreed, except for the replies not having the vlan tags. Cheers! -- Andrew Ruthven, Wellington, New Zealand At work: andrew.ruthven at catalyst.net.nz At home: andrew at etc.gen.nz GPG fpr: 34CA 12A3 C6F8 B156 72C2 D0D7 D286 CE0C 0C62 B791