Counter of the duplicated packets from a pcap file

[salwa17]

I want to count the duplicated packets from a pcap file. Duplicated packets are the packets whose the sequence number were already seen previously For that I extract firstly the list of the sequence number with this function:

  def seq_list(s):
    v = []
    a = [p['TCP'].seq if p.haslayer(TCP) else None
              for p in s]
    return a

Then I check if the current sequence number has already seen previously : (I'm not sure about this function)

   def is_dupl(s):
       v = seq_list(s)	
       a = []
       for p in s: 
          if p.haslayer(TCP):
              for i in range(0, len(v)):
                  a += v[0:i+1:1]
                  if p[TCP].seq in a:
                      return True 
          return False	

After that I made a function to get the list of the duplicated packet: (I'm not sure about this function)

    def find_dupl(s):
        tcpSeq = [p[TCP].seq if p.haslayer(TCP) and is_dupl(p) else None
                  for p in s]
        counter = Counter(tcpSeq)
        #del counter[None]
        print("---------------------length of counter dictionary : --", counter.items())
        return [[s[index] for index, seq in enumerate(tcpSeq)
            if seq == key]
            for (key, value) in counter.items()
            if value > 1]

And finally my counter:

    def duplication_pkt_count(s, s_ip, c_ip):
        sCount = 0
        cCount = 0
        duplication = find_dupl(s)

        for dup in duplication:
            for p in dup[1:]:
                if p.haslayer(IP):
                    if (p[IP].src == s_ip):
                        sCount += 1
                    if (p[IP].src == c_ip):
                        cCount += 1
        return (sCount, cCount)

And then I called the function duplication_pkt_count in my main.py code
The result is wrong. The number of the duplicated packets is wrong.

Please any help ?

[Gribouillis]

This looks horribly complicated. I may be wrong, but it seems to me that you simply want to sort the packets by their sequence number and then group consecutive packets having the same sequence numbers. Finally, remove the groups with only one packet. This is a common operation and it can be performed by combining the functions sorted() and itertools.groupby()

Tell us if this works any better

from itertools import groupby

def tcp_seq(p):
    return p['TCP'].seq

def groups_by_seq(s):
    x = sorted((p for p in s if p.haslayer('TCP')), key=tcp_seq)
    y = [(k, list(g)) for k, g in groupby(x, key=tcp_seq)]
    return [(seq, group) for seq, group in y if len(group) > 1]

def duplication_pkt_count(s, s_ip, c_ip):
    sCount = 0
    cCount = 0
 
    for seq, group in groups_by_seq(s):
        for p in group:
            if p.haslayer(IP):
                if (p[IP].src == s_ip):
                    sCount += 1
                if (p[IP].src == c_ip):
                    cCount += 1
    return (sCount, cCount)

[salwa17]

Thank you so much for your help.

Wow your solution is very simple.

My goal is like you described it but instead to remove the duplicated packet I want to count them. That's wht my duplication_pkt_count function has to return.

[Gribouillis]

My goal is like you described it but instead to remove the duplicated packet I want to count them. That's wht my duplication_pkt_count function has to return.

This function is difficult to understand, you could perhaps elaborate on what are s_ip and c_ip and what you're trying to count exactly.

[bowlofred]

You have different pieces in play. You should find out where the error is. Is find_dupl() broken, or is it okay and the problem is in duplication_pkg_count()?

First thing is that find_dupl looks odd to me. You're going to the trouble of making a nice Counter object, but then when you're done you're looping over it constantly. Much better it to just call into it.

I think instead of

    return [[s[index] for index, seq in enumerate(tcpSeq)
        if seq == key]
        for (key, value) in counter.items()
        if value > 1]

you could just do:

    return [s for s in tcpSeq if counter[s] > 1]

Also, maybe instead of 'counter' you could use a name like 'seq_count'.

[salwa17]

s_ip is the ip of the server and c_ip is of the client.
So if p[IP].src == s_ip i will count the duplicated packets coming from the server with sCount += 1 and if p[IP].src == c_ip i will count the duplicated packets coming from the client with cCount += 1


What I want to do is nearly what you describe previous but instead remove the duplicated packets, I retrieve them in a counter.

[bowlofred]

What is the point of is_dupl(). Why isn't just looking at the count in find_dupl() sufficient?

[Gribouillis]

What I want to do is nearly what you describe previous but instead remove the duplicated packets, I retrieve them in a counter.

I don't understand this part. My code doesn't remove any packet from anywhere. It just sorts them and groups them by the same sequence numbers.

[salwa17]

@ Gribouillis: Yes I know. I just re-explaining my need regarding your explanation: "Finally, remove the groups with only one packet"

So with your code, I got the whole of packets if I want only the duplicated one I have to check if my [tcp].seq is in this list. Something like this ? are you agree ?

def is_dupl(s):
    v = duplication_pkt_count(s)
    for p in s: 
        if p.haslayer(TCP) and  p.haslayer(IP): 
            for i in range(0, len(v)):
                if p[TCP].seq in v and (p[IP].src == s_ip):
                    sCount += 1
                if p[TCP].seq in v and (p[IP].src == c_ip):
                    cCount += 1
            return 	(sCount, cCount)