From wessels at measurement-factory.com Tue Aug 3 06:30:56 2010 From: wessels at measurement-factory.com (Duane Wessels) Date: Tue, 3 Aug 2010 00:30:56 -0600 (MDT) Subject: [dsc] Compile errors In-Reply-To: References: <1280348329.5768.2.camel@mcbain.mana.lan> Message-ID: On Wed, 28 Jul 2010, Stephan Lagerholm wrote: > Hi Duane and thanks for your response, > > The collector will not easily run on the DNS server in question but I > can get a pcap file from it that I can scp out. What will happen if I > hack the DSC source to read from a file instead? Will the statistics get > all screwed up? Yes they will. The collector uses the computer's clock (rather than pcap timestamps) to mark the 60-second boundaries and decide when it should make a new XML file. DW From wessels at measurement-factory.com Fri Aug 13 23:38:16 2010 From: wessels at measurement-factory.com (Duane Wessels) Date: Fri, 13 Aug 2010 17:38:16 -0600 (MDT) Subject: [dsc] New DSC release available Message-ID: Hello DSC users, I've posted a new version of DSC today, which you can find at http://dns.measurement-factory.com/tools/dsc/source.html Just a couple of noteworthy changes since the last posted release: - Peter Koch contributed a patch that allows the collector to read from pcap files on disk. - Alexander Mayrhofer contributed a patch that adds a country indexer if you have libGeoIP installed. See the CHANGES file at the download page for the less significant changes. DW From fines at macalester.edu Mon Aug 16 16:03:20 2010 From: fines at macalester.edu (Ted Fines) Date: Mon, 16 Aug 2010 11:03:20 -0500 Subject: [dsc] Problem with dsc on Ubuntu Linux In-Reply-To: <20100721174127.GE18955@macbook.catpipe.net> References: <20100721174127.GE18955@macbook.catpipe.net> Message-ID: Hi, I am seeing the same thing on CentOS Linux (2.6.18-164.11.1.el5). If I run it with "-d" it writes an .xml file and exits as planned, but if I just run it it terminates immediately with a malloc error. I reported this error on the list back in February: http://www.measurement-factory.com/pipermail/dsc/2010-February/000195.html Duane Wessels quickly offered a patch to remedy the problem, but it didn't work for me. I've tried again today, with the new SVN code, which also has the patch he suggested included. No go, it still doesn't work and fails the same way as before. I would like to hear from anyone who has been able to get dsc collector to run on CentOS, and exchange config / system info to figure this out. Thanks, Ted Fines Macalester College On Wed, Jul 21, 2010 at 12:41 PM, Phil Regnauld wrote: > Hi, > > Trying to setup DSC to work on Ubuntu Linux, but am seeing the collector > dying > immediately with a malloc error when trying to run it. See the output > further down. > > Note that the collector runs if -d is specified, and exits as planned after > the > first write. > > Version: dsc-200912101623 > OS: Ubuntu, Linux 2.6.31-14-generic-pae i686 > > The collector configuration is the dsc.conf.sample, with the following > changes: > > local_address 192.168.50.116; > run_dir "/usr/local/dsc/run/localhost"; > interface eth0; > > Here's the output of the crash. > > I can provide any additonal info as required. > > Thanks, > Phil > > > > # /usr/local/dsc/bin/dsc -f /usr/local/dsc/etc/dsc.conf > *** glibc detected *** /usr/local/dsc/bin/dsc: malloc(): memory corruption: > 0x08f97048 *** > ======= Backtrace: ========= > /lib/tls/i686/cmov/libc.so.6[0xb75aa0d1] > /lib/tls/i686/cmov/libc.so.6[0xb75accc3] > /lib/tls/i686/cmov/libc.so.6(__libc_malloc+0x58)[0xb75ae978] > /usr/lib/libstdc++.so.6(_Znwj+0x27)[0xb7782bb7] > /usr/lib/libstdc++.so.6(_ZNSs4_Rep9_S_createEjjRKSaIcE+0x66)[0xb775e436] > /usr/lib/libstdc++.so.6[0xb775f661] > /usr/lib/libstdc++.so.6(_ZNSsC1ERKSsjj+0x52)[0xb775f8d2] > /usr/local/dsc/bin/dsc[0x8064b03] > /usr/local/dsc/bin/dsc[0x805acf3] > /usr/local/dsc/bin/dsc[0x805ad0f] > /usr/local/dsc/bin/dsc[0x8053490] > /usr/local/dsc/bin/dsc[0x8053f37] > /usr/local/dsc/bin/dsc[0x8053f37] > /usr/local/dsc/bin/dsc[0x8053f37] > /usr/local/dsc/bin/dsc[0x80568b4] > /usr/local/dsc/bin/dsc[0x804fde0] > /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7555b56] > /usr/local/dsc/bin/dsc[0x804b031] > ======= Memory map: ======== > 08048000-0806f000 r-xp 00000000 08:01 40633 /usr/local/dsc/bin/dsc > 0806f000-08070000 r--p 00026000 08:01 40633 /usr/local/dsc/bin/dsc > 08070000-08071000 rw-p 00027000 08:01 40633 /usr/local/dsc/bin/dsc > 08071000-08112000 rw-p 00000000 00:00 0 > 08f66000-08fa8000 rw-p 00000000 00:00 0 [heap] > b7400000-b7421000 rw-p 00000000 00:00 0 > b7421000-b7500000 ---p 00000000 00:00 0 > b753d000-b753f000 rw-p 00000000 00:00 0 > b753f000-b767d000 r-xp 00000000 08:01 927 /lib/tls/i686/cmov/ > libc-2.10.1.so > b767d000-b767e000 ---p 0013e000 08:01 927 /lib/tls/i686/cmov/ > libc-2.10.1.so > b767e000-b7680000 r--p 0013e000 08:01 927 /lib/tls/i686/cmov/ > libc-2.10.1.so > b7680000-b7681000 rw-p 00140000 08:01 927 /lib/tls/i686/cmov/ > libc-2.10.1.so > b7681000-b7684000 rw-p 00000000 00:00 0 > b7684000-b76a0000 r-xp 00000000 08:01 2719 /lib/libgcc_s.so.1 > b76a0000-b76a1000 r--p 0001b000 08:01 2719 /lib/libgcc_s.so.1 > b76a1000-b76a2000 rw-p 0001c000 08:01 2719 /lib/libgcc_s.so.1 > b76a2000-b76c6000 r-xp 00000000 08:01 931 /lib/tls/i686/cmov/ > libm-2.10.1.so > b76c6000-b76c7000 r--p 00023000 08:01 931 /lib/tls/i686/cmov/ > libm-2.10.1.so > b76c7000-b76c8000 rw-p 00024000 08:01 931 /lib/tls/i686/cmov/ > libm-2.10.1.so > b76c8000-b77ae000 r-xp 00000000 08:01 2758 > /usr/lib/libstdc++.so.6.0.13 > b77ae000-b77b2000 r--p 000e6000 08:01 2758 > /usr/lib/libstdc++.so.6.0.13 > b77b2000-b77b3000 rw-p 000ea000 08:01 2758 > /usr/lib/libstdc++.so.6.0.13 > b77b3000-b77ba000 rw-p 00000000 00:00 0 > b77ba000-b77ca000 r-xp 00000000 08:01 2704 /lib/tls/i686/cmov/ > libresolv-2.10.1.so > b77ca000-b77cb000 r--p 00010000 08:01 2704 /lib/tls/i686/cmov/ > libresolv-2.10.1.so > b77cb000-b77cc000 rw-p 00011000 08:01 2704 /lib/tls/i686/cmov/ > libresolv-2.10.1.so > b77cc000-b77cf000 rw-p 00000000 00:00 0 > b77cf000-b77e2000 r-xp 00000000 08:01 934 /lib/tls/i686/cmov/ > libnsl-2.10.1.so > b77e2000-b77e3000 r--p 00012000 08:01 934 /lib/tls/i686/cmov/ > libnsl-2.10.1.so > b77e3000-b77e4000 rw-p 00013000 08:01 934 /lib/tls/i686/cmov/ > libnsl-2.10.1.so > b77e4000-b77e6000 rw-p 00000000 00:00 0 > b77e6000-b7817000 r-xp 00000000 08:01 39037 /usr/lib/libpcap.so.1.0.0 > b7817000-b7818000 r--p 00031000 08:01 39037 /usr/lib/libpcap.so.1.0.0 > b7818000-b7819000 rw-p 00032000 08:01 39037 /usr/lib/libpcap.so.1.0.0 > b781e000-b7820000 rw-p 00000000 00:00 0 > b7820000-b7821000 r-xp 00000000 00:00 0 [vdso] > b7821000-b783c000 r-xp 00000000 08:01 2446 /lib/ld-2.10.1.so > b783c000-b783d000 r--p 0001a000 08:01 2446 /lib/ld-2.10.1.so > b783d000-b783e000 rw-p 0001b000 08:01 2446 /lib/ld-2.10.1.so > bfe89000-bfe9e000 rw-p 00000000 00:00 0 [stack] > Aborted > > _______________________________________________ > dsc mailing list > dsc at measurement-factory.com > http://www.measurement-factory.com/mailman/listinfo/dsc > From fines at macalester.edu Tue Aug 17 14:41:15 2010 From: fines at macalester.edu (Ted Fines) Date: Tue, 17 Aug 2010 09:41:15 -0500 Subject: [dsc] Problem with dsc on Ubuntu Linux In-Reply-To: <20100721174127.GE18955@macbook.catpipe.net> References: <20100721174127.GE18955@macbook.catpipe.net> Message-ID: Hi, I have run into the same problem on CentOS 5--exactly the same behavior you describe. If run with "-d" it writes one .xml file and exits as planned. If run as you'd normally want it to, it immediately dies with the malloc error. I too posted this problem back in February: http://www.measurement-factory.com/pipermail/dsc/2010-February/000195.html Duane Wessels quickly replied and offered a modification to the Makefile.in to try, but it did not fix the problem for me. Other work became more pressing so I just gave up. Yesterday though I tried again with the latest version from svn. I noticed that the aforementioned patch is included in the source now. Still, it just doesn't work and crashes the same way with the same malloc error. So I brought up a CentOS 4.8 system, just to see if it would work on that. It uses libpcap 0.8.3 (CentOS 5.4 uses libpcap 0.9.4; numerous other libraries are different versions too). dsc runs perfectly fine on the CentOS 4.8 system, as it turns out! I then copied the compiled binary from the CentOS 4.8 system to the CentOS 5.4 system and tried running it. It immediately died, but with an error that it couldn't find the libpcap.so.0.8.3 library. That's understandable since there wasn't one. I created a symlink in /usr/lib for it (ln -s libpcap.so.0.9.4 libpcap.so.0.8.3), then tried running it again, hoping the good people at tcpdump.org would make 0.9.4 backward compatible with 0.8.3. Hey, it works! I've had dsc running overnight on my CentOS 5.4 system and it has been humming along producing loads of .xml files. Hooray! I realize this is a total kludge. But like Apache says, "Hey, it works!" I thought I'd post this as it might give Phil an idea to try something similar and might provide insight to Duane as to why we get the malloc problem. Hope this helps, Ted On Wed, Jul 21, 2010 at 12:41 PM, Phil Regnauld wrote: > Hi, > > Trying to setup DSC to work on Ubuntu Linux, but am seeing the collector > dying > immediately with a malloc error when trying to run it. See the output > further down. > > Note that the collector runs if -d is specified, and exits as planned after > the > first write. > > Version: dsc-200912101623 > OS: Ubuntu, Linux 2.6.31-14-generic-pae i686 > > The collector configuration is the dsc.conf.sample, with the following > changes: > > local_address 192.168.50.116; > run_dir "/usr/local/dsc/run/localhost"; > interface eth0; > > Here's the output of the crash. > > I can provide any additonal info as required. > > Thanks, > Phil > > > > # /usr/local/dsc/bin/dsc -f /usr/local/dsc/etc/dsc.conf > *** glibc detected *** /usr/local/dsc/bin/dsc: malloc(): memory corruption: > 0x08f97048 *** > ======= Backtrace: ========= > /lib/tls/i686/cmov/libc.so.6[0xb75aa0d1] > /lib/tls/i686/cmov/libc.so.6[0xb75accc3] > /lib/tls/i686/cmov/libc.so.6(__libc_malloc+0x58)[0xb75ae978] > /usr/lib/libstdc++.so.6(_Znwj+0x27)[0xb7782bb7] > /usr/lib/libstdc++.so.6(_ZNSs4_Rep9_S_createEjjRKSaIcE+0x66)[0xb775e436] > /usr/lib/libstdc++.so.6[0xb775f661] > /usr/lib/libstdc++.so.6(_ZNSsC1ERKSsjj+0x52)[0xb775f8d2] > /usr/local/dsc/bin/dsc[0x8064b03] > /usr/local/dsc/bin/dsc[0x805acf3] > /usr/local/dsc/bin/dsc[0x805ad0f] > /usr/local/dsc/bin/dsc[0x8053490] > /usr/local/dsc/bin/dsc[0x8053f37] > /usr/local/dsc/bin/dsc[0x8053f37] > /usr/local/dsc/bin/dsc[0x8053f37] > /usr/local/dsc/bin/dsc[0x80568b4] > /usr/local/dsc/bin/dsc[0x804fde0] > /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7555b56] > /usr/local/dsc/bin/dsc[0x804b031] > ======= Memory map: ======== > 08048000-0806f000 r-xp 00000000 08:01 40633 /usr/local/dsc/bin/dsc > 0806f000-08070000 r--p 00026000 08:01 40633 /usr/local/dsc/bin/dsc > 08070000-08071000 rw-p 00027000 08:01 40633 /usr/local/dsc/bin/dsc > 08071000-08112000 rw-p 00000000 00:00 0 > 08f66000-08fa8000 rw-p 00000000 00:00 0 [heap] > b7400000-b7421000 rw-p 00000000 00:00 0 > b7421000-b7500000 ---p 00000000 00:00 0 > b753d000-b753f000 rw-p 00000000 00:00 0 > b753f000-b767d000 r-xp 00000000 08:01 927 /lib/tls/i686/cmov/ > libc-2.10.1.so > b767d000-b767e000 ---p 0013e000 08:01 927 /lib/tls/i686/cmov/ > libc-2.10.1.so > b767e000-b7680000 r--p 0013e000 08:01 927 /lib/tls/i686/cmov/ > libc-2.10.1.so > b7680000-b7681000 rw-p 00140000 08:01 927 /lib/tls/i686/cmov/ > libc-2.10.1.so > b7681000-b7684000 rw-p 00000000 00:00 0 > b7684000-b76a0000 r-xp 00000000 08:01 2719 /lib/libgcc_s.so.1 > b76a0000-b76a1000 r--p 0001b000 08:01 2719 /lib/libgcc_s.so.1 > b76a1000-b76a2000 rw-p 0001c000 08:01 2719 /lib/libgcc_s.so.1 > b76a2000-b76c6000 r-xp 00000000 08:01 931 /lib/tls/i686/cmov/ > libm-2.10.1.so > b76c6000-b76c7000 r--p 00023000 08:01 931 /lib/tls/i686/cmov/ > libm-2.10.1.so > b76c7000-b76c8000 rw-p 00024000 08:01 931 /lib/tls/i686/cmov/ > libm-2.10.1.so > b76c8000-b77ae000 r-xp 00000000 08:01 2758 > /usr/lib/libstdc++.so.6.0.13 > b77ae000-b77b2000 r--p 000e6000 08:01 2758 > /usr/lib/libstdc++.so.6.0.13 > b77b2000-b77b3000 rw-p 000ea000 08:01 2758 > /usr/lib/libstdc++.so.6.0.13 > b77b3000-b77ba000 rw-p 00000000 00:00 0 > b77ba000-b77ca000 r-xp 00000000 08:01 2704 /lib/tls/i686/cmov/ > libresolv-2.10.1.so > b77ca000-b77cb000 r--p 00010000 08:01 2704 /lib/tls/i686/cmov/ > libresolv-2.10.1.so > b77cb000-b77cc000 rw-p 00011000 08:01 2704 /lib/tls/i686/cmov/ > libresolv-2.10.1.so > b77cc000-b77cf000 rw-p 00000000 00:00 0 > b77cf000-b77e2000 r-xp 00000000 08:01 934 /lib/tls/i686/cmov/ > libnsl-2.10.1.so > b77e2000-b77e3000 r--p 00012000 08:01 934 /lib/tls/i686/cmov/ > libnsl-2.10.1.so > b77e3000-b77e4000 rw-p 00013000 08:01 934 /lib/tls/i686/cmov/ > libnsl-2.10.1.so > b77e4000-b77e6000 rw-p 00000000 00:00 0 > b77e6000-b7817000 r-xp 00000000 08:01 39037 /usr/lib/libpcap.so.1.0.0 > b7817000-b7818000 r--p 00031000 08:01 39037 /usr/lib/libpcap.so.1.0.0 > b7818000-b7819000 rw-p 00032000 08:01 39037 /usr/lib/libpcap.so.1.0.0 > b781e000-b7820000 rw-p 00000000 00:00 0 > b7820000-b7821000 r-xp 00000000 00:00 0 [vdso] > b7821000-b783c000 r-xp 00000000 08:01 2446 /lib/ld-2.10.1.so > b783c000-b783d000 r--p 0001a000 08:01 2446 /lib/ld-2.10.1.so > b783d000-b783e000 rw-p 0001b000 08:01 2446 /lib/ld-2.10.1.so > bfe89000-bfe9e000 rw-p 00000000 00:00 0 [stack] > Aborted > > _______________________________________________ > dsc mailing list > dsc at measurement-factory.com > http://www.measurement-factory.com/mailman/listinfo/dsc > From regnauld at nsrc.org Tue Aug 17 15:54:17 2010 From: regnauld at nsrc.org (Phil Regnauld) Date: Tue, 17 Aug 2010 17:54:17 +0200 Subject: [dsc] Problem with dsc on Ubuntu Linux In-Reply-To: References: <20100721174127.GE18955@macbook.catpipe.net> Message-ID: <20100817155414.GC72333@macbook.catpipe.net> Ted Fines (fines) writes: > > I realize this is a total kludge. But like Apache says, "Hey, it works!" I > thought I'd post this as it might give Phil an idea to try something similar > and might provide insight to Duane as to why we get the malloc problem. Thanks Ted, this is great work. I think some APIs have changed across versions of libpcap, so there might be something there. I'll try your fix tonight and GDB step through the calls. Cheers, Phil From wessels at measurement-factory.com Mon Aug 23 23:26:17 2010 From: wessels at measurement-factory.com (Duane Wessels) Date: Mon, 23 Aug 2010 17:26:17 -0600 (MDT) Subject: [dsc] Problem with dsc on Ubuntu Linux In-Reply-To: References: <20100721174127.GE18955@macbook.catpipe.net> Message-ID: On Tue, 17 Aug 2010, Ted Fines wrote: > Yesterday though I tried again with the latest version from svn. I noticed > that the aforementioned patch is included in the source now. Still, it just > doesn't work and crashes the same way with the same malloc error. Hi Ted, I wonder if valgrind can help us here. Perhaps you can run it (with the crashy version) like this: sudo valgrind --trace-children=yes --log-file=/tmp/valgrind.log bin/dsc etc/dsc.conf DW From wessels at measurement-factory.com Mon Aug 23 23:49:44 2010 From: wessels at measurement-factory.com (Duane Wessels) Date: Mon, 23 Aug 2010 17:49:44 -0600 (MDT) Subject: [dsc] DSC futures meeting in Denver? Message-ID: Greetings DSC Users! In recent months I have been talking to some of you about what the future holds for DSC. I am not really in a position to give it a significant amount of time (and have not given it much attention for at least a year). In October there is a DNS-OARC meeting in Denver, Colorado, USA and I think that quite a few DSC users may be there. I am proposing to have a day (or perhaps 1/2 day) meeting of interested parties to discuss the future of DSC. Such a meeting would have an agenda like this: - How do we use DSC now and what do we like - What asspects of DSC should be updated/fixed/improved/removed? - Can we put a plan in place to make it happen? Note that I am assuming the DSC community generally feels that the software should be evolved for our future needs. If this assumption is incorrect, now would be a good time to let me know :-) To be clear: assuming we get to the final point in the above list, I will be asking you (your companies) to make financial contributions so that a programmer can be employed to implement the necessary changes. Details (such as amounts and where such money would be collected) are not known at this time. That will be part of the discussion. Please let me know if you think such a meeting is a good idea and whether or not you'd be able to attend. Duane W.