LessPythonInvoke

LessPython C# <-> Python Interop

A high-performance, cross-platform interop layer between C# and Python using shared memory (MMF) for data transfer and a lightweight TCP notify channel. Optimized for large NumPy arrays via a compact binary protocol and zero-copy views.

Quick start

1. Ensure Python is available (optionally in a venv) and numpy installed if you use array features.
2. Add this project to your solution or reference the produced DLL.
3. Write a Python module with functions to call from C#.
4. Initialize and call via a typed interface:

# exporting_module.py:
def add_ints(a, b):
    return int(a) + int(b)

public interface IMyPyApi
{
    [PythonImport] int add_ints(int a, int b);
    [PythonImport] NdArray inc_ndarray(NdArray nd);
    [PythonImport("inc_ndarray_timed")] (NdArray, double) inc_ndarray_timed(NdArray nd);
}

var api = LessPythonInvoke.Initialize<IMyPyApi>(
    pythonVenvPath: "",                 // optional venv root ("" => system python)
    exportingPyPath: "lesspy_work/exporting_module.py", // your module path
    debug: false,
    keepAlive: true,
    capacity: 256 * 1024 * 1024          // per-file MMF capacity
);

int x = api.add_ints(2, 40);

Initialization

LessPythonInvoke.Initialize<T>(string pythonVenvPath, string exportingPyPath, bool debug=false, bool keepAlive=false, int capacity=16*1024*1024) - pythonVenvPath: optional; empty string uses system python/python3. - exportingPyPath: path to your Python module file. - debug: enables extra logging from the shim. - keepAlive: keep Python server and MMFs running across C# process runs. - capacity: size of each MMF file (request/response). Must be large enough for your largest payload.

MMF/aux files are placed under bin/<config>/<tfm>/lesspy_work/: - req.mmf, resp.mmf: shared memory files - lesspy_shim.py: shim injected and launched - python.pid: Python PID (keepalive) - notify.port: TCP port (keepalive reconnect)

Attributes and binding

• Mark interface methods with [PythonImport]. The method name maps to the Python function name by default; override via [PythonImport("py_name")].
• Supported argument/return types:
  • int, long, float, double, bool, string, byte[]
  • int[], int[][] (nested)
  • ValueTuple with supported element types
  • NdArray (NumPy) with shape/dtype/data

NdArray

NdArray represents a NumPy array: Shape, Dtype, and data. On responses, if Python returns a NumPy array, the C# side returns an NdArray backed by a zero-copy view over resp.mmf (DataMemory). Accessing NdArray.Data materializes a copy only on demand.

KeepAlive

• When keepAlive: true, the Python process keeps running after C# exits.
• On next run, C# will reuse the existing Python server by reading python.pid and notify.port, then connect over TCP.
• If the notify connection drops, Python keeps listening; subsequent runs will reconnect.

Performance

• Binary protocol (no JSON) over SHM, single-byte notify via localhost TCP.
• Direct pointer copies for MMF read/write; no intermediate payload buffers on C# side.
• Zero-copy ndarray responses (views over resp.mmf).
• Example (4096x4096 float32 inc): ~24.9 ms end-to-end for one un-timed call (~5.1 GB/s raw read+write).

Tips: - Use larger capacity for big transfers. - Prefer returning arrays directly from Python for zero-copy on C#. - For transforms, compute into the response buffer on Python side to avoid extra copies.

Troubleshooting

• MMF locked on rapid restarts: you may see user-mapped section open. Retry after a moment; add exponential backoff if needed.
• Type mismatch in tuples: ensure element types align (e.g., float vs double). The invoker coerces primitives where needed.
• KeepAlive not reusing: ensure python.pid, notify.port, and req.mmf/resp.mmf remain present and Python is still running.
• Capacity exceeded: increase capacity to fit your largest request/response.