Puzzling import issue that I have no idea how to solvr

[starseeker]

I'm building an API with django ninja, for design reasons I'm using some logic inside microservice/logic/one_module.py that needs to get JSON data from microservice/service/repository.py

This is my project structure:

Correlator/microservice/   <-------- Root directory is Correlator and that's where my CWD points from terminal
|   .coveragerc
|   manage.py
|   pytest.ini
|   
+---microservice/
    |   asgi.py
    |   models.py
    |   settings.py
    |   urls.py
    |   wsgi.py
    |   __init__.py
    |   
    +---logic/
    |       config.py
    |       correlate.py
    |       correlation_rules.py
    |       generate_ticket.py
    |       send_ticket.py
    |       startup.py  <--------- Code is being executed here
    |       temp.py
    |       __init__.py
    |   
    |           
    +---service/
    |   |   application.py
    |   |   domain.py
    |   |   repository.py   <--------- Data is here (must be imported)
    |   |   __init__.py
    |   |   
    |   +---schemas/
    |   |       application.py
    |   |       domain.py
    |   |       repository.py
    |   |       __init__.py
    |   |       
    |   +---tests/
    |       |   test_application.py
    |       |   test_domain.py
    |       |   test_logic.py
    |       |   test_repository.py
    |       |   __init__.py
    |       |   
    |       +---json/
    |               alert_1.json
    |               client_1.json
    |               secrets_1.json
    |           
    +---static/
    |       logo.png
    |       script.js
    |       styles.css
    |       
    +---templates/
            base.html
            email.html

In service/repository.py I have:

json_call = requests.get(...etc...)
clients, alerts, secrets = ...comprehension to get the data as dict...

In logic/startup.py I have:

from microservice.service.repository import clients, secrets, alerts

clients = ...some code using the data...

Running it like that, it raises an error:

Traceback (most recent call last):
  File "...windows_path...\Projects\Correlator\microservice\microservice\logic\startup.py", line 1, in <module>
    from microservice.service.repository import clients, secrets, alerts
ModuleNotFoundError: No module named 'microservice'

VSCode (in Windows 11) finds it just fine, however, I may also try a relative import:

from ..service.repository import clients, secrets, alerts

clients = ...some code using the data...

Running it like that, it raises an error:

Traceback (most recent call last):
  File "...windows_path...\Projects\Correlator\microservice\microservice\logic\startup.py", line 1, in <module>
    from ..service.repository import clients, secrets, alerts
ImportError: attempted relative import with no known parent package

VSCode does find it too with no issues. It also redirects to __init__.py correctly when trying to inspect it with the IDE. I'll copy a print(sys.path) statement so you know what's going on with that.

['...windows_path...\\Projects\\Correlator\\microservice\\microservice\\logic', '...windows_path...\\AppData\\Local\\Programs\\Python\\Python312\\python312.zip', '...windows_path...\\AppData\\Local\\Programs\\Python\\Python312\\DLLs', '...windows_path...\\AppData\\Local\\Programs\\Python\\Python312\\Lib', '...windows_path...\\AppData\\Local\\Programs\\Python\\Python312', '...windows_path...\\Projects\\Correlator\\venv', '...windows_path...\\Projects\\Correlator\\venv\\Lib\\site-packages']

Trying to use the sys.path.append (I want to avoid that at all costs, I know writing some import code in __init__.py could be the solution) doesn't work either, will paste code and traceback.

import sys
import pathlib
import os
print("CWD WITH OS --->",os.getcwd())
sys.path.append(str(pathlib.WindowsPath().cwd()))
print("PYTHONPATH ---> ", sys.path)
from microservice.service.repository import clients, secrets, alerts

CWD WITH OS ---> ...windows_path...\Projects\Correlator\microservice\microservice\logic

PYTHONPATH --->  ['...windows_path...\\Projects\\Correlator\\microservice\\microservice\\logic',
'...windows_path...\\AppData\\Local\\Programs\\Python\\Python312\\python312.zip',
'...windows_path...\\AppData\\Local\\Programs\\Python\\Python312\\DLLs',
'...windows_path...\\AppData\\Local\\Programs\\Python\\Python312\\Lib',
'...windows_path...\\AppData\\Local\\Programs\\Python\\Python312',
'...windows_path...\\Projects\\Correlator\\venv', '...windows_path...\\Projects\\Correlator\\venv\\Lib\\site-packages'],
'...windows_path...\\Projects\\Correlator\\microservice\\microservice\\logic']

Traceback (most recent call last):
  File "...windows_path...\Projects\Correlator\microservice\microservice\logic\startup.py", line 1, in <module>
    from microservice.service.repository import clients, secrets, alerts
ModuleNotFoundError: No module named 'microservice'

I tried to do a test-A.py and test-B.py with foo() and var:int inside /logic and /service but same issue, no luck solving it. Honestly don't know what is causing the error, maybe it's a Django thing, every directory has an empty __init__.py

Many thanks, I'm running the /logic/startup.py script from the base path ...Correlator/ in Windows 11 with Python 3.12

[Pedroski55]

I would just check the paths first, to see if what you seek is actually in the path you are using.

I don't use Windoze so I am not familiar with the paths.

I keep my home-made modules in '/home/pedro/myPython/myModules'

from pathlib import Path
 
# set starting directory
# winpath = Path("C:\Correlator")
path2modules = Path('/home/pedro/myPython/myModules')
# if you have various subdirectories, you can loop through them
# I don't have subdirectories!
dirs = [d for d in path2modules.iterdir() if d.is_dir()]
# this to find all files in your path
path2modules = Path('/home/pedro/myPython/myModules') 
filelist = [filename for filename in path2modules.iterdir() if filename.is_file()]
for filename in filelist:
    print(f"\nfilename: {filename.name}")
    print(f"file suffix: {filename.suffix}")
    print(f"full path: {filename.resolve()}")
    print(f"filepath parts: {filename.parts}")

Example output:

Output:
filename: makeWebpage.py
file suffix: .py
full path: /home/pedro/myPython/myModules/makeWebpage.py
filepath parts: ('/', 'home', 'pedro', 'myPython', 'myModules', 'makeWebpage.py')

I see sometimes people use r' before the path. Not sure why. In case of funny characters or spaces perhaps.

dest = r'C:\Test'

Or you can specifically search for any particular file and, if found, print its full path.

[starseeker]

yup, already checked, it's not the problem

maybe it's a python Import bug, no idea

the problem is in the import itself, not the path usage or anything like that. I'm trying to import some (dict) variables from ./dir_1/module_1.py to ./dir_2/module_2.py

[deanhystad]

You cannot do this:

from microservice.service.repository import clients, secrets, alerts
 
clients = ...some code using the data...

This code creates a variable named "clients" in the local module. If you want to modify clients in microservice.service.repositiory you need to do something like this.

import microservice.service.repository as repository
 
repository.clients = ...some code using the data...

This is a scope problem, not a path problem. Assignment creates local variables unless you force Python to make them elsewhere. In a function you can use "global" to tell python that a vairable inside a function is a module variable. There is no "global" for variables in other modules, so you need to use the module object when assigning the variable. You can see this more easily in my simplified example:
file repository.py

x = 5

def pprint():
    print(x)

file service.py

import repository
from repository import x, pprint

x = 42
pprint()
repository.x = 42
pprint()

Output:
5
42