Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s2: Do 2 overlapping match checks #839

Merged
merged 1 commit into from
Jul 27, 2023
Merged

s2: Do 2 overlapping match checks #839

merged 1 commit into from
Jul 27, 2023

Conversation

klauspost
Copy link
Owner

@klauspost klauspost commented Jul 27, 2023

Mainly faster on long matches, but has little to no regression on short matches.

λ s2c -cpu=1 --bench=5 \compress\github-june-2days-2019.json

BEFORE: Compressing... 6273951764 -> 944876986 [15.06%]; 9.43s, 634.5MB/s
AFTER:  Compressing... 6273951764 -> 944876986 [15.06%]; 9.269s, 645.5MB/s

λ s2c -cpu=1 --bench=5 \compress\nyc-taxi-data-10M.csv
BEFORE: Compressing... 3325605752 -> 884713054 [26.60%]; 8.296s, 382.3MB/s
AFTER:  Compressing... 3325605752 -> 884713054 [26.60%]; 8.173s, 388.1MB/s

λ s2c -cpu=1 --bench=5 \compress\enwik9
BEFORE: Compressing... 1000000000 -> 416582393 [41.66%]; 3.706s, 257.3MB/s
AFTER:  Compressing... 1000000000 -> 416582393 [41.66%]; 3.636s, 262.3MB/s

Mainly faster on long matches, but has little to no regression on short matches.
@klauspost klauspost merged commit c1dcc38 into master Jul 27, 2023
18 checks passed
@klauspost klauspost deleted the matchlen-do-16 branch July 27, 2023 13:26
kodiakhq bot referenced this pull request in cloudquery/filetypes Oct 1, 2023
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [github.com/klauspost/compress](https://togithub.com/klauspost/compress) | indirect | minor | `v1.16.7` -> `v1.17.0` |

---

### Release Notes

<details>
<summary>klauspost/compress (github.com/klauspost/compress)</summary>

### [`v1.17.0`](https://togithub.com/klauspost/compress/releases/tag/v1.17.0)

[Compare Source](https://togithub.com/klauspost/compress/compare/v1.16.7...v1.17.0)

#### What's Changed

-   Add dictionary builder by [@&#8203;klauspost](https://togithub.com/klauspost) in [https://github.com/klauspost/compress/pull/853](https://togithub.com/klauspost/compress/pull/853)
-   Add xerial snappy read/writer by [@&#8203;klauspost](https://togithub.com/klauspost) in [https://github.com/klauspost/compress/pull/838](https://togithub.com/klauspost/compress/pull/838)
-   flate: Add limited window compression by [@&#8203;klauspost](https://togithub.com/klauspost) in [https://github.com/klauspost/compress/pull/843](https://togithub.com/klauspost/compress/pull/843)
-   s2: Do 2 overlapping match checks by [@&#8203;klauspost](https://togithub.com/klauspost) in [https://github.com/klauspost/compress/pull/839](https://togithub.com/klauspost/compress/pull/839)
-   flate: Add amd64 assembly matchlen by [@&#8203;klauspost](https://togithub.com/klauspost) in [https://github.com/klauspost/compress/pull/837](https://togithub.com/klauspost/compress/pull/837)
-   gzip: Copy bufio.Reader on Reset by [@&#8203;thatguystone](https://togithub.com/thatguystone) in [https://github.com/klauspost/compress/pull/860](https://togithub.com/klauspost/compress/pull/860)
-   zstd: Remove offset from bitReader by [@&#8203;greatroar](https://togithub.com/greatroar) in [https://github.com/klauspost/compress/pull/854](https://togithub.com/klauspost/compress/pull/854)
-   fse, huff0, zstd: Remove always-nil error returns by [@&#8203;greatroar](https://togithub.com/greatroar) in [https://github.com/klauspost/compress/pull/857](https://togithub.com/klauspost/compress/pull/857)
-   tests: unnecessary use of fmt.Sprintf by [@&#8203;testwill](https://togithub.com/testwill) in [https://github.com/klauspost/compress/pull/836](https://togithub.com/klauspost/compress/pull/836)
-   tests: Fix OSS fuzzer t.Run by [@&#8203;klauspost](https://togithub.com/klauspost) in [https://github.com/klauspost/compress/pull/852](https://togithub.com/klauspost/compress/pull/852)
-   tests: Use Go 1.21.x by [@&#8203;klauspost](https://togithub.com/klauspost) in [https://github.com/klauspost/compress/pull/851](https://togithub.com/klauspost/compress/pull/851)

#### New Contributors

-   [@&#8203;testwill](https://togithub.com/testwill) made their first contribution in [https://github.com/klauspost/compress/pull/836](https://togithub.com/klauspost/compress/pull/836)
-   [@&#8203;thatguystone](https://togithub.com/thatguystone) made their first contribution in [https://github.com/klauspost/compress/pull/860](https://togithub.com/klauspost/compress/pull/860)

**Full Changelog**: klauspost/compress@v1.16.7...v1.17.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on the first day of the month" (UTC), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi4xMDkuNCIsInVwZGF0ZWRJblZlciI6IjM2LjEwOS40IiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant