Correct py got error on another windows 7

[meetinnet]

So sorry that I can't describe accurately in the title because I don't really understand this issue.
I have a simple Py file which simply post some data to a website.It worked very well on my Windows 10 and windows 7, but got strange error on another windows 7. Part of the codes:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import requests
url = 'https://someweb.com/submit'

headers={
        "Host": "something.com",
        "Connection": "keep-alive",
        "Content-Length": "100",
        
}
   
def format(Cookie):
    cookies = {}
    for line in Cookie.split(";"):
            if line.find("=") != -1:  
                name, value = line.strip().split("=")
                cookies[name] = value
    return cookies

fp = open ("1.txt","r")
Cookie = fp.read()
fp.close()
CookieNew = format(Cookie)
DataNew = '{"vId":"000104","serviceId":"1002"}'

requests.post(url, data=DataNew, headers=headers, cookies=CookieNew)

The format(cookie) function changes the format to JSON I guess because the website only accept this format.(Googled)
The first error is

Error:
UnicodeEncodeError: 'latin-1' codec can't encode characters in position xx-xx:
 ordinal not in range(256)

Then I googled and changed the

fp = open ("1.txt","r")

to

fp = open ("1.txt","r",encoding = "utf-8-sig")

Now I got this error from website:

{"timestamp":"2020-04-23 16:47:15","status":500,"error":"Internal
Server Error","exception":"org.springframework.http.converter.HttpMessageNotRead
ableException","message":"Could not read document: Unexpected character ('i' (co
de 105)): was expecting comma to separate OBJECT entries\n at [Source: java.io.P
ushbackInputStream@6bc0aae7; line: 1, column: 105]; nested exception is com.fast
erxml.jackson.core.JsonParseException: Unexpected character ('i' (code 105)): wa
s expecting comma to separate OBJECT entries\n at [Source: java.io.PushbackInput
Stream@6bc0aae7; line: 1, column: 105]","path":"/dir/cou/submit"}

I'm confused that this is the same py file. What makes this error and how do I google this because it looks like java responses.

If I use encoding = "utf-8" instead of encoding = "utf-8-sig", the error is

Error:
UnicodeEncodeError: 'latin-1' codec can't encode character '\ufeff' in position
xx: ordinal not in range(256)

The OS of windows 10 and the widnows 7 with py error are Asia version,the other windows7 without error is English version.

The content of 1.txt here,it's a cookie file.

Output:
Hm_lvt_4a693f348daf8481dd541c844122c053=1585791910; JSESSIONID=D524881BB65001A53B445AA4EDC14FC7; Hm_lpvt_4a693f348daf8481dd541c844122c053=1587654145; Hm_lvt_bc864c0a0574a7cabe6b36d53206fb69=1583560881; 

Thanks for help and sorry for the chaotic description.

[Larz60+]

You probably still don't have the right codec.
I have used chardet2 with great success: https://pypi.org/project/chardet2/
Simple to use, from command line, it's chardet2 <filename>

[meetinnet]

You probably still don't have the right codec.
I have used chardet2 with great success: https://pypi.org/project/chardet2/
Simple to use, from command line, it's chardet2 <filename>

Thank you. I tried chardet2. Command line doesn't work so I inserted it to python.

fp = open ("1.txt","r")
Cookie = fp.read()
fp.close()
code = chardet.detect(Cookie)
print(code)

Error:
File "C:\Python37\site-packages\chardet\__init__.py", line 34, in detect
    '{0}'.format(type(byte_str)))
TypeError: Expected object of type bytes or bytearray, got: <class 'str'>

[Larz60+]

Traceback shows that you didn't supply filename.

usage: if filename = myfile.txt,command: chardet2 myfile.txt

example (from command line):

Output:
$ chardet results.txt
results.txt: ascii with confidence 1.0

[meetinnet]

Traceback shows that you didn't supply filename.

usage: if filename = myfile.txt,command: chardet2 myfile.txt

example (from command line):

Output:
$ chardet results.txt
results.txt: ascii with confidence 1.0

The same way on windows system ? But I got this "is not recognized as an internal or external command" after runinng "E:\Python\chardet2 cookie.txt" which I don't think it's a regular command.

Installing collected packages: chardet2
  Running setup.py install for chardet2 ... done
Successfully installed chardet2-2.0.3

[Larz60+]

It needs to be installed:
pip install chardet2

[meetinnet]

It needs to be installed:
pip install chardet2

Hi,my previous post was the process of pip install chardet2.

[snippsat]

It's called chardetect on Windows,when used from command line.

E:\div_code
λ chardetect my.csv
my.csv: ascii with confidence 1.0

Not sure if that would help on this problem.

UnicodeEncodeError: 'latin-1' codec can't encode character '\ufeff' in position

\ufeff indicate that this file is saved with BOM(Byte order mark).

So usually should encoding='utf-8-sig' fix this.

>>> f = open('file', mode='r')
>>> f.read()
'\ufefftest'

>>> f = open('file', mode='r', encoding='utf-8-sig')
>>> f.read()
'test'

Not sure when you say Asia version of Windows that this happen on,what version of Python
Try in a errors='replace' in open to see what is replace,there is also errors="ignore"

open ("1.txt", "r", encoding="utf-8-sig", errors='replace')

[meetinnet]

Thanks for the time. I searched the whole disk and finally found the chardetect in a deep directory C:\Users\Administrator\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\Scripts
On the win10 and win7 system which worked fine the chardetect result is E:\Python\Cookie.txt: ascii with confidence 1.0, on that Win7 which has problem,the chardetect result is UTF-8-SIG with confidence 1.0.

Before I got chardetect I was not sure which format of cookie.txt I used on that win7 machine.(save as to utf-8 and ANSI using notepad many times), now I checked the cookie.txt again using chardetect on the win10 machine, it's ascii and works fine then I copied it to win7 machine and checked again for sure.
Now that error message is only below whatever using encoding="utf-8" or "utf-8-sig".

Error:
{"timestamp":"2020-04-26 00:41:12","status":500,"error":"Internal
Server Error","exception":"org.springframework.http.converter.HttpMessageNotRead
ableException","message":"Could not read document: Unexpected character ('i' (co
de 105)): was expecting comma to separate OBJECT entries\n at [Source: java.io.P
ushbackInputStream@1962cd72; line: 1, column: 105]; nested exception is com.fast
erxml.jackson.core.JsonParseException: Unexpected character ('i' (code 105)): wa
s expecting comma to separate OBJECT entries\n at [Source: java.io.PushbackInput
Stream@1962cd72; line: 1, column: 105]","path":"/dir/cou/submit"}