Copy data from 1 blk device in machine A to another blk device in machine B

[AbhishekV]

Eg:
copy data from /dev/sdc1 of Machine A and write to /dev/sdb1 of Machine B.
To achive this, i have written socket_client.py and socket_server.py. I put all my affort, but i could not achieve the result.
In the current code, i want that a socket_client.py should read block device data and send to socket_server.py with the offest value from where data has been read. In socket program i just want to unpack the data contained in struct and display on the screen.

Note:
1) Data returned after unpack will be returned in tuple.
2) My code contains lots of debugging lines. Please let me know if you need any other details.

Please let me know if there is any better way to do this. I am new to pyhon. So as per my understanding i tried to write this code to achive the result.
Need help from all of you.

Socket_client.py

#!/usr/bin/python3
import socket
import sys
import os,pickle
import struct as st
dev_read_data = os.open("/dev/sdc1",os.O_RDONLY)
buffer_size = 230400 #1048576
offset = 0
#creating a tcp/ip socket
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)

server_address = ('localhost',10000)
print ('connecting to {} port {}'.format(server_address[0],server_address[1]))
sock.connect(server_address)
data_list = []
sent_data_size = 0
try:
    while True:
        data = os.pread(dev_read_data,buffer_size,offset)
        #Data should be binary as its string
        packed_data = st.pack("%dsi" %(len(data),),data,offset)
        st_size = st.calcsize("%dsi" %(len(data)))
        print ('st_size {}'.format(st_size))
        packed_st_size = st.pack("i",st_size)
        sock.send(packed_st_size)
        #Sending data
        while sent_data_size < st_size: #Send data till sent_data_size < st_size i.e. struct size
            print ('{} size expected to recev'.format(st_size))
            print ('value of sent_data_size before send command {}'.format(sent_data_size))
            sock.send(packed_data)
            sent_data_size = sock.recv(16) # rec the amount of data sent from socket_client.py to socket_server.py. This info is sent by socket_server.py
            print ('value of sent_data_size sock.send command {}'.format(sent_data_size))

            sent_data_size = st.unpack("Q",sent_data_size) #unpacking sent_data_size so that it can be compared in while loop condition
            print ('value of sent_data_size sock.send command {}'.format(sent_data_size))
            print ('Inside while loop')
        #print ('Sent from offset {} data size {}'.format(offset,buffer_size))
        print ('Outside while loop')
        offset += buffer_size # Changing the offset to read new data using pread.
        if offset == 10036977152:
            break

finally:
    print ('Closing socket')
    sock.close()

Socket_server.py

#!/usr/bin/python3
import sys
import socket
import os
import pickle
import struct as st
#create a tcp/ip socket
dev_write_data = os.open("/dev/sdb1",os.O_WRONLY)
buffer_size = 230400 #1048576

sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
server_address = ('localhost',10000)
print ('Starting up on {} port {}'.format(server_address[0],server_address[1]))
sock.bind(server_address)
sock.listen(5)
data_size = 0
while True:
    print ('waiting for a connection')
    connection,client_address = sock.accept()
    try:
        print ('connection from {}'.format(client_address))
        packed_st_size = connection.recv(16) #rec size of struct sent from socket_client.py
        print ('Struct size {}'.format(packed_st_size))
        st_size = st.unpack("i",packed_st_size)[0] # After unpacking data returned is tuple. So this stmt takes 0th index data from tuple and initializing to st_size
        print ('Outside while loop struct size {}'.format(st_size))
        print ('Outside while Data_Size {}'.format(data_size))
        while data_size <= st_size:             #Keep looping till rec data_size  <= st_size i.e size of struct which was actually sent.
            print ('Inside while loop struct size {}'.format(st_size))
            print ('Outside while Data_Size {}'.format(data_size))
            if data_size == 0: # if data_size is 0, data variable is initialized with data send by socket_client.py
                data = connection.recv(st_size)
            else:
                data += connection.recv(st_size) #Otherwise byte stream data is concatinated with sequentially sent data by socket_client.py
            x = len(data) #get data len.
            data_len_rec = st.pack("Q",x) #get the data len rec in bytes using struct pack method
            connection.send(data_len_rec) #Sending the len of data recv to socket_client.py
            print ('{} bytes received.'.format(x))
            data_size += len(data) # Adding length of rec data to data_size for iteration
        #final_data = st.unpack("%dsi" %(len(data)),data)
        #print ('Final Data recv {}'.format(final_data))
    finally:
        connection.close()

[Gribouillis]

What happens when you run it? The only detail I see is that you could use sock.sendall() because sock.send() is not guaranteed to send the whole data, see the doc.

Unrelated to this, you could configure the server socket with

sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

to ease the port reuse between server shutdowns.

[DeaD_EyE]

An implementation which uses a buffer and memoryview:

import socket


host = '127.0.0.1'
port = 1337

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# use TCP for data transmission
sock.connect((host, port))
buffer = bytearray(1024**2) # 1MiB
view = memoryview(buffer)
with open(file, 'rb') as fd:
    while True:
        readlen = fd.readinto(buffer)
        # buffer is reused
        if not readlen:
            break
        sock.send(view[:readlen]
        # sends read data from memoryview to socket
    sock.close()

The buffer is reused for the chunks. The method readinto of a fileobject, puts the data into the buffer and returns the amount of data, which has been written to the buffer. The memoryview is just a method to have fast access on slices.
The socket object has also the function sendfile which takes a fileobject. Maybe this is the fastest implementation. I never tried it.