From daniel at pch.net  Fri May  3 04:17:34 2013
From: daniel at pch.net (Daniel Griggs)
Date: Fri, 3 May 2013 13:17:34 +0900
Subject: [dsc] Performance Issues Qname and Client Subnet
Message-ID: <86835742-56D9-4A18-9E04-4AEDCEF98D5E@pch.net>


I have compiled the latest version of DSC from the svn repository on CentOS 5 and am running into an issue.

I have configured the following rules;

dataset qtype dns All:null Qtype:qtype queries-only;
dataset rcode dns All:null Rcode:rcode replies-only;
dataset opcode dns All:null Opcode:opcode queries-only;
dataset qname dns All:null Qname:qname queries-only;
dataset clientsub dns All:null ClientSubnet:client_subnet queries-only;
dataset addr_qname dns Qname:qname ClientSubnet:client_subnet queries-only;

Running without the last rule it hovers at around 8% CPU and very little memory (less than 10Megs) for the packet capturing processing and amount of time spend processing the captured data to the text file in the spin off process is less than a second. But when I add in the final rule the CPU hovers around 16-12%, the memory usage grows to over 200Megs, that's fine, but when it comes to writing the results out to disk it becomes painfully slow, pegging the CPU at 100% and taking minutes to complete, only writing kb's a second to the disk. In total taking just over 5 minutes and the final file is 10Mbs.

I assume from the way it uses memory that it collating the data as it goes so I am confused as to why it's taking so long to express the data into XML.

Any ideas?
Is this a regression, expected behaviour or a new interesting problem?

--
Daniel




From daniel at pch.net  Sun May  5 01:37:18 2013
From: daniel at pch.net (Daniel Griggs)
Date: Sun, 5 May 2013 10:37:18 +0900
Subject: [dsc] Performance Issues Qname and Client Subnet
In-Reply-To: <86835742-56D9-4A18-9E04-4AEDCEF98D5E@pch.net>
References: <86835742-56D9-4A18-9E04-4AEDCEF98D5E@pch.net>
Message-ID: <F75E77E7-75BE-498D-8186-67501E664BB7@pch.net>

Hi all,

Does it make a performance difference to it if the all of the local ip's are not listed?

On 3/05/2013, at 1:17 PM, Daniel Griggs <daniel at pch.net> wrote:

> 
> I have compiled the latest version of DSC from the svn repository on CentOS 5 and am running into an issue.
> 
> I have configured the following rules;
> 
> dataset qtype dns All:null Qtype:qtype queries-only;
> dataset rcode dns All:null Rcode:rcode replies-only;
> dataset opcode dns All:null Opcode:opcode queries-only;
> dataset qname dns All:null Qname:qname queries-only;
> dataset clientsub dns All:null ClientSubnet:client_subnet queries-only;
> dataset addr_qname dns Qname:qname ClientSubnet:client_subnet queries-only;
> 
> Running without the last rule it hovers at around 8% CPU and very little memory (less than 10Megs) for the packet capturing processing and amount of time spend processing the captured data to the text file in the spin off process is less than a second. But when I add in the final rule the CPU hovers around 16-12%, the memory usage grows to over 200Megs, that's fine, but when it comes to writing the results out to disk it becomes painfully slow, pegging the CPU at 100% and taking minutes to complete, only writing kb's a second to the disk. In total taking just over 5 minutes and the final file is 10Mbs.
> 
> I assume from the way it uses memory that it collating the data as it goes so I am confused as to why it's taking so long to express the data into XML.
> 
> Any ideas?
> Is this a regression, expected behaviour or a new interesting problem?
> 
> --
> Daniel
> 
> 
> _______________________________________________
> dsc mailing list
> dsc at measurement-factory.com
> http://www.measurement-factory.com/mailman/listinfo/dsc

--
Daniel