Well, there is the -py3 option that adds annotations to the proxy classes.

But they are very useless for IDEs because they use the C/C type, and not the actual python type.

IDE like VS code only support auto-completion based on PEP 484 type hints.
@wsfulton Is there any plan to support Python3 type hinting in SWIG release 4.0.0 or earlier?

These type annotations would be a great addition to SWIG. SWIG is an all volunteer project, but I don't know of anyone working on this. I encourage anyone interested in adding this support to discuss it here.

Hi,
As long as there is no such feature, is it possible to avoid the generation of the non-PEP 484-compatible annotations produced by -py3 ?

For instance, when producing sphinx documentation with the sphinx_autodoc_typehints extension, the actual annotations produce errors such as :

Exception occurred:
  File "/usr/lib64/python3.7/typing.py", line 448, in __init__
    raise SyntaxError(f"Forward reference must be an expression -- got {arg!r}")
  File "<string>", line None
SyntaxError: Forward reference must be an expression -- got 'std::string'

(the argument has been type-annotated with std::string)

Currently the -py3 option adds in Python3 function annotations that use the C type names. The "autodoc" feature uses the Python type names mostly, but sometimes the C type when documenting types. If my understanding of PEP-484 is correct, a different set of types are required (those in the typing module).

We need a consistent and flexible approach for the 3 different sort of types across docstrings, doxygen sourced comments and Python3 function annotations. Looks like these are the 3 types possible:

1. Pure Python types
2. The C types
3. The types from the typing module required fro PEP-484

The doxygen sourced types come from one of the C type or the "doctype" typemap and autodoc uses one of the C type or "docstring:name". Needless to say this is bit of a mess. A proposal to clean this all up is most welcome, if not urgently required. I strongly suggest a clear proposal is put into a new issue for discussion. Unfortunately the merging in of the Doxygen module for SWIG-4.0.0 is about to go live, I'm wondering whether we should clean this up before pushing out SWIG-4.0.0, despite the long delays in releasing it. Any thoughts?

Note that this PEP still explicitly does NOT prevent other uses of annotations, nor does it require (or forbid) any particular processing of annotations, even when they conform to this specification.

I'd take this to mean that whatever tools you are using should work without PEP-484 compliant type hints and they shouldn't require PEP-484 type hints. Nevertheless, SWIG should have more control over whether or not they should be generated, with or without -py3.

In my project these type hints were not acceptable for users, so I whipped up a script to rename C/C names into interpretable Python types. Hope this is helpful to people out there!

https://github.com/krishauser/improve_swig_docs

The first stage of this has been implemented in 2072ae1 by removing the current C/C type annotations from being generated with the -py3 option. The implementation has been designed to be expandable and configurable for alternative implementations. The three options in the list above could for example be implemented using:

1. Pure Python types: �ature("python:annotations", "python");
2. C/C types: �ature("python:annotations", "c");
3. Typing module types in PEP-484: �ature("python:annotations", "typing");

with 2. now done.

I think pure Python types and typing module types are the same. In Python 3.10 it is often possible to type everything without a single import from the typing module.

Since PEP 585 (Python 3.8) generics can be type hinted in standard collections:

a: list[str] = ["a", "b", "c"]
b: dict[str, int] = {"a": 1, "b": 2, "c": 3}

Since PEP 604 (Python 3.10) union types can be written as X | Y:

a: str | int = "abc"
b: str | int = 123

It is currently also recommended to not use Optional and instead use:

a: str | None = None

This might be further improved by PEP 645 (Draft) which would allow writing optional types as x?:

a: str? = None

I am interested in adding PEP 484 type hints to SWIG. Below is some relevant info and a proposal of how this could be achieved:

Python Compatibility

Inline Annotations

Annotations can be added to the Python source files themselves, this requires a new syntax and is not backwards compatible. The function annotation syntax was added to Python 3.0 in PEP 3107 and the variable annotation syntax was added to Python 3.6 in PEP 526.

Valid since Python 3.0:

def func(parameter: "parameter annotation") -> "return annotation":
    pass

Valid since Python 3.6:

var: "variable annotation" = 123

Stub Files (.pyi)

Annotations can alse be added through a separate stub file, see PEP 484. These stub files are meant for type checkers only and are not loaded at runtime. This means that they are backwards compatible, they can always use the latest syntax and they can be used to annotate Python 2 code. Stub files are also the only way to type compiled modules.

Inline annotations are generally preferred over stub files because they are more readable. With stub files the annotations have to be read from a separate file or from IDE tooltips.

Terminology

This is not official, it's just what I use:

• Annotation
  • def f(a: "whatever") -> "whatever" function annotations as introduced in PEP 3107
  • var: "whatever" variable annotations as introduced in PEP 526
  • Not necessarily type hints, may be docs for example, any string is allowed.
• Type hint
  • An annotation that gives an indication of a variable/parameter/return type, like the existing SWIG C/C based type hints.
• PEP 484 type hint
  • "Official" type hints that follow the guidelines outlined in PEP 484 and later PEPs extending it.

SWIG Python Type Hints Proposal

Phase 1

The first step is to complete the existing inline annotations to prepare for Python 4 and projects only supporting compatible Python versions (3.6 ):

• Complete inline variable annotation support using existing SWIG C/C based type hints
  • variableWrapper
  • constantWrapper
  • nativeWrapper
  • staticmembervariableHandler
  • memberconstantHandler
• Add a way to control inline annotations
  • Maybe �ature("python:annotations", "inline_c");?

Phase 2

The second step is to add support for stub files:

• Basically write inline annotations to a stub file
• Add support for typing overloads
• Add support for typing compiled modules (_example.pyi for _example.so)
• Add a way to control stub file generation, this should be mutually exclusive with inline annotations
  • Maybe �ature("python:annotations", "stub_c");?

Phase 3

The third step is to add support for PEP 484 type hints:

• Add helper that can translate SwigTypes to Python types
• Add a way to manually set Python types (only if necessary)
• Add a way to specify which types should be generated (C/C based or Python)
  • Maybe for C/C based type hints
    • �ature("python:annotations", "inline_c");
    • �ature("python:annotations", "stub_c");
  • And for Python type hints
    • �ature("python:annotations", "inline");
    • �ature("python:annotations", "stub");

@bonqra, today I finished off the rest of what I wanted to contribute in this space by polishing off the variable annotations support in #1951. This basically implements your Phase 1. I suggest you tweak your plan slightly according to how this has been implemented. I'll come back with comments on the two remaining phases when done.

@wsfulton there are still some parts missing but adding them in will be easy thanks to the new variableAnnotation helper.
I've added the places that are missing variable annotations to Phase 1.

I think pure Python types and typing module types are the same.

While I am not very familiar with PEP 484 typing hints, they don't look the same to me. Consider this:

�ature("python:annotations", "c");

%include <std_vector.i>

%template(VectorInt) std::vector<int>;
void takeVector(std::vector<int> vi) {}
// [Edited to remove int <-> std::string confusion for the vector type]

I would see the 3 types as this:

1. Pure Python types
   def takeVector(vi: "VectorInt") -> "void":
2. C/C types:
   def takeVector(vi: "std::vector< int,std::allocator< int > >") -> "void":
3. Typing module PEP 484:
   def takeVector(vi: "vector[int]") -> "void":

I'm not sure what the typing module hints would exactly look like, but the container/generic syntax along the lines of container_name[type] seems to be the right way. This is not the same as what is shown in 1. In fact I suggest the proposal is enhanced to detail how to map C/C types to typing module type hints. In particular the STL container types and even user-defined containers.

Phase 1 is indeed missing some variable wrappers, such as wrapping global variables. I'm not sure how this would be implemented without stub files as the implementation of 'cvar' is in C code. I'm not sure how this would be achieved as there is no obvious C api to add in annotations.

Regarding Phase 2, is it not possible to have typing overloads implemented independent of stub files?

�ature("python:annotations", "c");

%include <std_vector.i>

%template(VectorInt) std::vector<int>;
void takeVector(std::vector<int> vi) {}
// [Edited to remove int <-> std::string confusion for the vector type]

The pure Python as well as the PEP 484 type hint for this would be def takeVector(vi: VectorInt) -> None:.

In fact I suggest the proposal is enhanced to detail how to map C/C types to typing module type hints. In particular the STL container types and even user-defined containers.

I will write up a detailed description but this might take some time. The gist of it is to use the correct fundamental type (None instead of void etc.) and for everything else use the generated Python classes. If there are exceptions to this they need to be handled but I first need to look at this closely to be able to determine what these exceptions might be.

Phase 1 is indeed missing some variable wrappers, such as wrapping global variables. I'm not sure how this would be implemented without stub files as the implementation of 'cvar' is in C code. I'm not sure how this would be achieved as there is no obvious C api to add in annotations.

Type hinting the wrappers only is fine for consumers of libraries, they would get the correct type hints. Long term it would be desirable to type hint compiled modules with stub files so the SWIG generated Python files aren't full of type errors.

But Phase 1 is only about adding as much type hints through inline annotations as possible, Phase 2 extends this through stub files to type hint everything so Phase 3 can enable full PEP 484 support in one go.

Regarding Phase 2, is it not possible to have typing overloads implemented independent of stub files?

They can be inline, too, but that comes with compatibility concerns. It would definitely be worth it to add them to the inline type hints. I will look into what version would be required (I think 3.7 but I'm not sure edit: it's available since 3.5). This could also be handled with another feature flag which would take care of any concerns about compatibility. Should I add it to Phase 1?

I'm not sure how this would be implemented without stub files as the implementation of 'cvar' is in C code.

This can be done without stub files:

/* Some global variable declarations */
%inline %{
extern int              ivar;
extern char            *strvar;
%}

class CvarInterface:
    ivar: int
    strvar: str

cvar: CvarInterface = _example.cvar

The pure Python as well as the PEP 484 type hint for this would be def takeVector(vi: VectorInt) -> None:.

Ah yes, of course, my bad, None would be used instead of void.

Regarding typing overloads. Yes, does feel like it belongs in Phase 1. I suggest break each stage down into sub-stages and the typing overloads would be a final sub-stage. It will really help reviewing and subsequent merging into master if you have a number of clearly defined bite size pull requests rather than one monster sized pull request.

Compatibility for typing overloads can indeed have their own feature flag to turn them off, how about �ature("python:annotations:nooverload")

Add a way to control inline annotations

Maybe �ature("python:annotations", "inline_c");?

This is currently �ature("python:annotations", "c"); I think the type of type hint should remain as is and we add another feature flag to control whether the type hint goes to the stub file or inline. Maybe �ature("python:annotations:stubfile") to use a stubfile instead of the default inline. So for C/C type hints to go to a stubfile, it would be:

�ature("python:annotations", "c");
�ature("python:annotations:stubfile");

Presumably the stub file name does not need to be configurable, it will always be the same as the module name but with a .pyi extension.

When you have your detailed proposal for PEP 484 type hints, I think it would be useful if took our classic example in Examples/python/class and hand wrote the final output and include in the proposal as an example. This example is remarkably similar to one discussed in Covariance and contravariance in PEP 484. Another one including the std::vector<int> mentioned earlier would also be an important one as it covers the STL.

The pure Python as well as the PEP 484 type hint for this would be def takeVector(vi: VectorInt) -> None:.

I think I see this now. However to me, it still seems that there is a difference between pure Python type hints and the typing module type hints. For VectorInt, the class definition would remain as is for pure Python type hints:

class VectorInt(object):
  ...

but would be enhanced for the typing module type hints, something along the lines of (I've probably got some of this wrong, but I can see similar examples in the PEP 484 doc):

from typing import List
class VectorInt(object, List[int]):
  ...

or maybe

from typing import Iterable, Container, Generic
class VectorInt(object, Iterable[int], Container[int], Generic[int]):
  ...

noting that the following code works for my SWIG example above, that is, both lists and tuples can be used instead of VectorInt:

takeVector([10, 20, 30])
takeVector((10, 20, 30))
takeVector(VectorInt((10, 20, 30)))

Regarding typing overloads. Yes, does feel like it belongs in Phase 1.

Compatibility for typing overloads can indeed have their own feature flag to turn them off, how about �ature("python:annotations:nooverload")

Perfect, I'll add them to Phase 1.

I suggest break each stage down into sub-stages and the typing overloads would be a final sub-stage. It will really help reviewing and subsequent merging into master if you have a number of clearly defined bite size pull requests rather than one monster sized pull request.

Yeah, I planned on doing each phase as one PR but splitting them up further makes sense.

Maybe �ature("python:annotations:stubfile") to use a stubfile instead of the default inline.

Sounds good, note that compiled modules will always be typed with a stub file.

Presumably the stub file name does not need to be configurable, it will always be the same as the module name but with a .pyi extension.

Correct.

When you have your detailed proposal for PEP 484 type hints, I think it would be useful if took our classic example in Examples/python/class and hand wrote the final output and include in the proposal as an example. This example is remarkably similar to one discussed in Covariance and contravariance in PEP 484. Another one including the std::vector<int> mentioned earlier would also be an important one as it covers the STL.

Great idea, the pyright playground doesn't support multiple files so I'll also set up a repo that can be explored with VSCode's "strict" Python analysis mode (which uses pyright under the hood).

However to me, it still seems that there is a difference between pure Python type hints and the typing module type hints.

I think I understand what you mean now, but I don't understand the need for this distinction. Why are pure Python type hints required? Couldn't all type hints be typing module type hints?

takeVector([10, 20, 30])
takeVector((10, 20, 30))
takeVector(VectorInt((10, 20, 30)))

I didn't consider that any sequence can be passed, the corrected type hint is:

def takeVector(vi: VectorInt | Sequence[int]) -> None: ...

Note that this can't be typed with pure Python type hints. It would be possible to do:

def takeVector(vi: VectorInt | list[int] | tuple[int]) -> None: ...

But this doesn't allow custom sequences.

And the typing module type hint of VectorInt would be:

from typing import Iterable, MutableSequence
# since 3.9 these are aliases to 
# from collections.abc import Iterable, MutableSequence
class VectorInt(MutableSequence[int]):
  ...

See collections.abc - Abstract Base Classes for Containers.

Manually typed examples are now available at https://github.com/bonqra/swig-python-type-hint-example.

The proposal isn't updated yet, I'll post again once that's done.

Hi @bonqra, any news yet?

here's what I did to get type annotations for return types. I'm on �ature(autodoc,0) but should more or less work for other levels too and I'm pretty sure it doesn't break stuff haha . maybe it helps someone

import sys
from pathlib import Path
import re

wrapper_file = sys.argv[1]
if os.path.exists(wrapper_file):
    print(os.path.basename(wrapper_file))

wrapper_text = Path(wrapper_file).read_text()
doc_pattern = re.compile('r"""([a-zA-Z_]*). ?(?=->)-> (.*)"""')
class_separator = "\nclass "
classes = wrapper_text.split(class_separator)
classes_annotated = []
for c in classes:
    for m in re.finditer(doc_pattern, c):
        print(f' method name: {m.group(1)} \t \t return type {m.group(2)}')
        c = re.sub(f'(def {m.group(1)}. ?(?=:))', f'\\1 -> "{m.group(2)}"', c, count=1)
    classes_annotated.append(c)

output = class_separator.join(classes_annotated)
open(wrapper_file, 'w ').close()
with open(wrapper_file, 'w ') as f:
    f.write(output)

My shot at the renaming of the "C style" return types, with a basic support of the std::vector wrapped classes :

#!/bin/bash

# This script convert "useless" C-style return types annotations created by Swig to ones pointing to the real python objects
# Supports "classic" objects and std::vector wrapped object

# Vector classes are expected to be defined as such: 
#   - %include "std_vector.i" present in the .i file
#   - namespace std { %template(VectorClassname) vector<namespace::Classname>; }

# Tested with Swig v4.1.1:
#   - with the �ature("python:annotations", "c"); parameter set in the .i file
#   - "-c   -python" command line arguments

# Usage:
#   - update the namespace and vector_classes parameters
#   - run swig on the <project>.i file
#   - run "bash rename_swig_types.sh <project>.py"

namespace="whatever"
declare -A vector_classes
# vector_classes["Classname"]="VectorClassname"
vector_classes["Class"]="ClassVector"

# First rename "classic" return types from -> "namespace::ClassName" to -> ClassName
sed -i -E "s/\"$namespace::(\w ) *\"/ \1/g" $1

# Then loop on all vector classes and rename all the std::vector stuff
for key in "${!vector_classes[@]}"
do
    sed -i "s/\"std::vector< $namespace::$key >::size_type\"/int/g" $1
    sed -i "s/\"std::vector< $namespace::$key >::difference_type\"/int/g" $1
    sed -i "s/\"std::vector< $namespace::$key,std::allocator< $namespace::$key > > *\"/${vector_classes[$key]}/g" $1
    sed -i "s/\"std::vector< $namespace::$key >::value_type const &\"/$key/g" $1
    sed -i "s/\"std::vector< $namespace::$key >::value_type\"/$key/g" $1
done

# Rename default types
sed -i 's/"int"/int/' $1
sed -i 's/"bool"/bool/' $1
sed -i 's/"float"/float/' $1
sed -i 's/"double"/float/' $1
sed -i 's/"std::string const &"/str/' $1

# insert annotations import to avoid errors with classes which return their own type
sed -i "1i\\from __future__ import annotations" $1

I'm using the -doxygen flag (which overrides autodoc; you can't use both at once) and settled on just writing a Python script to modify the SWIG-generated .py files to add type annotations. Return type annotations are easy since SWIG adds an :rtype: line in the docstring comment, I just had to do some simple parsing to go from :py:class:MyClass (there are `` symbols around MyClass there, but they mess up the formatting) or MyNamespace::MyClass to the valid Python equivalent.

For parameter type annotations I think I'll have to modify my C Doxygen comments to indicate type which is a shame but not really that big of a deal. After I do that, it'll again be some simple string parsing.

I'm really only concerned with the annotations so I can get type hinting popups in IDEs. A single Python script that I can set to run automatically after I run my SWIG makefile isn't really a big deal, though having official support would be nice.

If anyone has suggestions for a better way, I'm all ears. In the meantime, anyone looking for type hints with -doxygen should know that there isn't that much involved in writing a Python script to just parse and add them yourself.