Add Support for Specifying Headers in context.enqueue_links
Method
#894
Labels
enhancement
New feature or request.
t-tooling
Issues with this label are in the ownership of the tooling team.
When using the
context.enqueue_links
method, there is currently no way to specify custom headers (e.g., cookies or other required headers) for the links being enqueued. Some websites require specific headers or cookies to properly retrieve data from detail pages. It would be helpful to add support for passing headers or request options to context.enqueue_links to handle such cases.Example use case:
This feature would greatly enhance flexibility when crawling websites that require authentication or specific headers for access.
Thank you!
The text was updated successfully, but these errors were encountered: