How to make `import` statements to import from another path?

I want to "hack" python's import so that it will first search in the path I specified and fallback to the original if not found.

The traditional way of using sys.path will not work if there is a __init__.py, see below.

CASE 1: The following works:

Files:

.
├── a
│   ├── b.py       # content: x='x'
│   └── c.py       # content: y='y'
├── hack
│   └── a
│       └── b.py   # content: x='hacked'
└── test.py        # content: see below

# test.py
import sys
sys.path.insert(0, 'hack') # insert the hack path

from a.b import x
from a.c import y

print(x, y)

Running test.py gives hacked y as desired, where x is hacked :)

CASE 2: However, if there is a __init__.py in a, it will not work.

Files:

.
├── a
│   ├── b.py
│   ├── c.py
│   └── __init__.py # <- NOTE THIS
├── hack
│   └── a
│       └── b.py
└── test.py

Running test.py gives x y, where x is not hacked :(

CASE 3: To fix case 2, I tried adding __init__.py to the hack path, but this disables the fallback behavior.

Files:

.
├── a
│   ├── b.py
│   ├── c.py
│   └── __init__.py
├── hack
│   └── a
│       ├── b.py
│       └── __init__.py # <- NOTE THIS
└── test.py

Running test.py raises the following error as there is no c.py in the hack path and it fails to fallback to the original path.

ModuleNotFoundError: No module named 'a.c'

My question is, how to make case 2 work?

Additional background:

The above cases are just simplified examples. In the real situation,

• Both a and hack/a are large repos with many subfolders and files.

• The imported modules (e.g. a.b) may also contain import statements that need to be hacked.

Therefore, ideally the solution would be to only add a few lines of code at the top of test.py rather than modifying exisiting code.

UPDATE: I have come up with a solution below (I cannot accept my own answer in 2 days). If you have better solutions or suggestions, please feel free to discuss.

Instead of modifying sys.path one alternative approach is to use a helper function that tries importing from the given module path prefixed with hack. first before falling back to the actual given path upon an ImportError:

from operator import attrgetter

def import_from(from_path, *names):
    try:
        module = __import__(f'hack.{from_path}', fromlist=names)
    except ImportError:
        module = __import__(from_path, fromlist=names)
    return attrgetter(*names)(module)

x = import_from('a.b', 'x')
y = import_from('a.c', 'y')

The solution is to overwrite the default __import__ function (which is used by import statements) so that it first tries to import from the hack folder.

__import = __import__ # save the original

def _import(name, *a, **b):
    try:
        return __import('hack.'+name, *a, **b)
    except ImportError:
        return __import(name, *a, **b)

__builtins__.__import__ = _import # overwrite with our own