I have posted my question in stackoverflow but hasn't got any help yet.
https://stackoverflow.com/questions/5318...ng-python3
I need to sync any Postgres table from the latest copy of our production database (source) to a developer database without wiping out its testing data. The below simplified code works for most tables, but not for those with jsonb fields, due to the psycopg2.ProgrammingError: can't adapt type 'dict'
https://stackoverflow.com/questions/5318...ng-python3
I need to sync any Postgres table from the latest copy of our production database (source) to a developer database without wiping out its testing data. The below simplified code works for most tables, but not for those with jsonb fields, due to the psycopg2.ProgrammingError: can't adapt type 'dict'
import psycopg2
from psycopg2 import sql
tb = "table_to_be_copied"
##############################
# load data from source DB
##############################
conn_source = psycopg2.connect(host='source_localhost',
dbname=postgres,
user='xyz',
port=source_port)
cursor_source = conn_source.cursor()
cursor_source.execute(
sql.SQL("SELECT * from {}").format(sql.Identifier(tb))
)
# obtain column names on the fly for any given table
colnames = tuple([desc[0] for desc in cursor_source.description])
# jsonb's type code is 3802. This will help the program determine on the fly
# which columns are in jsonb.
typecodes = tuple([desc[1] for desc in cursor_source.description])
# obtain production data to be synced
rows = cursor_source.fetchall()
cursor_source.close()
conn_source.close()
##############################
# upsert data into destination DB
##############################
conn_dest = psycopg2.connect(host='dest_localhost',
dbname='postgres',
user='xyz',
port=dest_port)
cursor_dest = conn_dest.cursor()
for row in rows:
cursor_dest.execute(
sql.SQL("INSERT INTO {} ({}) VALUES ({}) \
ON CONFLICT (id) DO UPDATE SET ({}) = ({})").format(
sql.Identifier(tb),
sql.SQL(', ').join(map(sql.Identifier, colnames)),
sql.SQL(', ').join(sql.Placeholder() * len(colnames)),
sql.SQL(', ').join(map(sql.Identifier, colnames)),
sql.SQL(', ').join(sql.Placeholder() * len(colnames))),
row * 2)
conn_dest.commit()
cursor_dest.close()
conn_dest.close()
print ("Sync done")It'd be terrific if you could help answer this question. If not, I would still greatly appreciate if you can vote for my post at stackoverflow so to attract more reads. Thanks!
