feat: update/add pageSeparator params to LlamaParseReader #1051
Conversation
🦋 Changeset detectedLatest commit: 790daee The changes in this PR will be included in the next version bump. This PR includes changesets to release 9 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
commit: @llamaindex/autotool
@llamaindex/cloud
@llamaindex/community
@llamaindex/core
@llamaindex/env
@llamaindex/experimental
llamaindex
@llamaindex/wasm-tools
|
| @@ -20,7 20,7 @@ Copy the URL in your browser and select the server you want your bot to join. | |||
| #### DiscordReader() | |||
|
|
|||
| - `discordToken?`: The Discord bot token. | |||
| - `makeRequest?`: Optionally provide a custom request function for edge environments, e.g. `fetch`. See discord.js for more info. | |||
| - `requestHandler?`: Optionally provide a custom request function for edge environments, e.g. `fetch`. See discord.js for more info. | |||
There was a problem hiding this comment.
why this change in the llamaparse PR?
There was a problem hiding this comment.
I just noticed it and it's just a change in the docs, I mistakenly called it makeRequest in my previous PR, which is not the actual param name.
There was a problem hiding this comment.
@KindOfAScam Great catch! Thanks a lot. To avoid confusion, better send a separate PR next time
| @@ -41,7 41,9 @@ They can be divided into two groups. | |||
| - `doNotCache?` Optional. Set to true to not cache the document. | |||
| - `fastMode?` Optional. Set to true to use the fast mode. This mode will skip OCR of images, and table/heading reconstruction. Note: Non-compatible with `gpt4oMode`. | |||
| - `doNotUnrollColumns?` Optional. Set to true to keep the text according to document layout. Reduce reconstruction accuracy, and LLMs/embeddings performances in most cases. | |||
| - `pageSeparator?` Optional. The page separator to use. Defaults is `\\n---\\n`. | |||
| - `pageSeparator?` Optional. A templated page separator to use to split the text. If it contain `{page_number}`, it will be replaced by the next page number. If not set the default separator `\\n---\\n` will be used. | |||
There was a problem hiding this comment.
Why does the pageSeparator contain the {page_number}?
There was a problem hiding this comment.
Sorry, I just copied the python description.
I assume it's currently only relevant for JSON mode where a page_number is returned.
Improved the descriptions.
| @@ -207,6 213,15 @@ export class LlamaParseReader extends FileReader { | |||
| vendor_multimodal_api_key: this.vendorMultimodalApiKey, | |||
| }; | |||
|
|
|||
| // Filter out invalid specific parameters | |||
| const filteredParams = this.filterSpecificParams(LlamaParseBodyParams, [ | |||
There was a problem hiding this comment.
where is this used?
There was a problem hiding this comment.
Oh yeah.. I did forget to actually use it.
No description provided.