Python wrapper for the /users/notes endpoint of Sharkey (and probably also Misskey). You can use this to crawl the public posts of a user.

You can install the package via poetry (or another tool of your choosing):

poetry add sharkey-crawler

You can find the documentation at https://hexafuchs.github.io/sharkey-crawler/

from sharkey_crawler import SharkeyServer

SharkeyServer('example.org').user_notes(
    user_id='xxxxxxxxxx',
    allow_partial=True, 
    with_channel_notes=True,
    with_renotes=False,
    with_replies=False,
    with_files=False,
    limit=10,
    since_id=None,
    since_date=None,
    until_id=None,
    until_date=None
)

Checkout the docstring or the docs for more usage information.

# All
./venv/bin/pytest -m ""

# Unit
./venv/bin/pytest -m "unit"

# Integration
./venv/bin/pytest -m "integration"

# Unit and Integration
./venv/bin/pytest -m "integration or unit"

python3 -m venv venv
./venv/bin/python -m pip install --upgrade pip
./venv/bin/python -m pip install flit
./venv/bin/flit install --only-deps --deps develop

./venv/bin/tox

Make sure you have installed the dependencies.

cd docs
make clean
make html

You can find the documentation under docs/_build/html/index.html

Either add the dependency to the optional dependencies, or create a new dependency within the [project] namespace, e.g.:

[project]
...
dependencies = [
    "requests==2.32.3"
]

Then, install dependencies with flit:

./venv/bin/flit install --only-deps --deps develop
# or: ./venv/bin/flit install --only-deps --deps all

You might be asking yourself why this project does not expose more endpoints. It could, and it likely will, but currently, the endpoints are not well documented and it takes a lot of effort to even add a single endpoint, which Sharkey has a lot of. Since Sharkey is not very old and the future is still unclear, I will not take the effort it takes to support more endpoints until I have a use case for it or I see great demand. If you want more endpoints, there are two recommended solutions for this:

• open a discussion, so I and possibly other developers can see which endpoints are requested a lot of have an interesting use case
  • also, vote for endpoints you want to see added in the future
• contribute the endpoints yourself

There might also be solutions to automate parts of the development like creating Pydantic classes. If you are interested in this, feel free to contribute or open a discussion to organize this.

Please see CHANGELOG for more information on what has changed recently.

The MIT License (MIT). Please see License File for more information.