分享一个修改版的Shadowsocks，兼谈我们对猫鼠游戏的一点想法

作者： Anonymous, Anonymous
日期： 2022年10月15日，星期六
English version: Sharing a modified Shadowsocks as well as our thoughts on the cat-and-mouse game

我们在这篇文章中发布和开源一个修改版的Shadowsocks。这个版本的Shadowsocks可以绕过当前GFW的检测以及封锁。我们首先介绍这个修改后的Shadowsocks的原理，再分享一个如何部署服务器和客户端的简单教程。我们还会介绍其他两种当前能够帮助Shadowsocks和VMess绕过当前GFW封锁的办法。在文章的最后，我们分享我们对审查这个猫鼠游戏的一些看法。我们将从审查者每次开始大规模封锁的时间点选择出发，探讨审查者相比反审查社区缺乏灵活性这一根本性的弱点，并附上我们对如何利用审查者的弱点来达到四两拨千斤的效果的一些建议。

动机

我们在此时发表这个版本有三个目的：

1. 首先，我们想为中国网民提供一个（暂时）可行的翻墙方案。用增加翻墙协议多样性的方式，缓解10月3号以来的GFW对多种翻墙工具的大规模封锁。

2. 其次，我们想抛砖引玉地引起研究者和开发者的讨论。我们实证性的研究显示，当前的GFW已经可以精准地识别Shadowsocks，VMess，以及Obfs4这类完全加密协议（full-encrypted protocol）。我们估算GFW当前的流量检测算法会误伤约0.6%的非翻墙链接，而假阴性则低得几乎可以忽略不计。这迫切的需要我们群力群策地改进当前的协议。

3. 最后，我们想把这次发布当作一场实验，同时观察审查者和反审查社区在面对新的（反）审查事件时的反应速度。

为什么这个修改后的Shadowsocks可以规避GFW当前的检测和封锁？

我们与其他研究人员合作发现，当前的GFW会利用多种不同的规则来识别Shadowsocks，VMesss，以及Obfs4这类完全加密协议。其中一条规则就利用了这些加密流量的0比特与1比特的比例接近1:1的特性。因此，如果我们在加密流量中加入更多的0或1，再对比特序列进行重排，就可以达到改变原有比例特征，绕过检测和封锁的目的。

我怎么用这个修改版的Shadowsocks？

这个修改版的Shadowsocks基于Shadowsocks-rust，我们还利用Shadowsocks-android编译了apk文件供安卓用户使用。所有的客户端和服务端软件都可以在这个branch和这个release找到。

安装服务端

安装修改版的服务器的过程与安装任何其他Shadowsocks-rust服务器无异。

1. 首先你需要登陆你的远程服务器，然后下载编译好的文件：

wget https://github.com/gfw-report/shadowsocks-rust/releases/download/v0.0.1-beta/shadowsocks-v1.15.0-alpha.9.x86_64-unknown-linux-gnu.tar.xz
tar xvf shadowsocks-v1.15.0-alpha.9.x86_64-unknown-linux-gnu.tar.xz

2. 接着你需要创建一个配置文件：

sudo nano server_config.json

将下面的配置文件复制粘贴。注意，你需要把里面的ExamplePassword替换成一个更强的密码。你可以用以下命令在终端生成一个强密码：openssl rand -base64 16。你也应该考虑更改服务器端口8388。

{
  "server": "0.0.0.0",
  "server_port": 8388,
  "password": "ExamplePassword",
  "method": "aes-256-gcm"
}

将上方替换过密码的配置信息复制粘贴到配置文件后， 按Ctrl x退出。 退出时，文本编辑器将问你"Save modified buffer?"，请输入y然后按回车键。

3. 你现在就可以运行二进制的服务器了。但是为了让它在你退出SSH后还能继续运行，你可以考虑建立一个tmux会话：

tmux

然后再运行服务器：

./ssserver -c ./server_config.json

最后，按Ctrl b再按d就可以脱离tmux会话了。

防火墙配置

我们使用ufw来管理Shadowsocks服务器的防火墙。

在基于Debian的服务器上，可以通过如下命令安装ufw：

sudo apt update && sudo apt install -y ufw

然后开放有关ssh和Shadowsocks-rust的端口。 请注意，以下命令假设你在server_config.json中的server_port的值为8388。 如果你的server_port用了其他的值，请对以下命令作相应的修改：

sudo ufw allow ssh
sudo ufw allow 8388

现在我们启动ufw:

sudo ufw enable

启动时如果弹出Command may disrupt existing ssh connections. Proceed with operation (y|n)?，请输入y并按回车键。

最后，请用sudo ufw status检查一下你的配置是否和下面的一样：

Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
8388                       ALLOW       Anywhere
22/tcp (v6)                ALLOW       Anywhere (v6)
8388 (v6)                  ALLOW       Anywhere (v6)

客户端配置

下面是桌面版客户端的配置文件，记得server的值替换为你远程服务器的IP地址。如果你们是用了我们提供的安卓apk在手机上使用，那么配置就和往常的使用Shadowsocks-android的办法一样。

{
    "server": "ExampleServerIP",
    "server_port": 8388,
    "password": "ExamplePassword",
    "method": "aes-256-gcm",
    "local_address": "127.0.0.1",
    "local_port": 1080
}

不足

• 因为我们对Shadowsocks协议做了修改，因此暂不兼容其他Shadowsocks客户端和服务端。用户需要下载我们准备的客户端和服务端。
• 目前客户端只支持Windows，Linux，macOS，安卓手机，和安卓电视版。不支持iOS。我们欢迎有能力的iOS开发者们实现兼容这个修改的协议。也欢迎iOS开发者与我们联系，我们将分享其他绕过审查的办法供你们参考。
• 在加密方式上，修改版还不支持Shadowsocks-2022。这并非我们刻意不支持Shadowsocks-2022，只是时间和精力有限，还没来得及支持。我们推荐选择：chacha20-ietf-poly1305或者aes-256-gcm。
• 在实现方式上，我们直接在核心代码上进行了修改。这并非我们想要另起炉灶，维护一个Shadowsocks协议的分支版本，只是时间和经历有限，还没有把修改后的算法做成一个可供用户选择的选项。我们将与Shadowsocks开发者积极沟通，希望最终能以提供给用户选项的方式，将新协议合并到Shadowsocks中。

你们还知道什么其他的可以规避当前封锁的方法吗？

我们还知道两种目前可行的方案，他们都利用了另一种不同的GFW流量检测规则：

如果你是V2Ray用户，你可以开启ExperimentReducedIvHeadEntropy选项来避免GFW的检测和封锁。这个方案的好处是你无须在服务器进行任何修改。

如果你是Shadowsocks用户，@database64128还实现了另外一种绕过审查的办法。因为对协议做了修改，所以需要同时更新客户端和服务端。

分享我们对猫鼠游戏的一点看法

我们想借此机会分享一些我们对于审查与反审查这个猫鼠游戏的看法。这些看法既来自于我们对审查者和反审查社区两方的观察与思考，又同时受到klzgrad，David Fifield以及其他许多许多反审查者的深深影响。

审查时间点的选择

长期关注互联网审查的朋友们一定已经发现，中国的审查者总是在政治敏感事件来临前的数天或数周开始大规模地使用他们的新审查武器。实际上，这样的时间点选择绝非偶然，那具体的原因是什么呢？我们猜想至少有三点，也欢迎大家继续讨论补充：

1. 首先，对审查者而言，保证至少在政治敏感的时间段内，对社会舆论和信息流通有着足够的控制，是一项重要的政治任务。这项任务通常会被官方话语描述为"为某某期间的网络安全护航"。

2. 其次，审查者在政治敏感时期更能容忍审查造成的误伤，因此给新的审查武器投入使用提供了更宽松的试错环境。Tschantz等人对大量审查事件进行分析总结后发现，真实世界的审查者通常宁可少封，也要尽量避免错封带来的影响（见recommendation 5）。而这种倾向仅在政治敏感时期略有变化：审查者会变得更能容忍误伤的代价，以达到其所追求的社会管控。而在这时上线新的武器，即使有前期没有测试出来的错误造成误伤或事故，也更能被容忍。klzgrad在这篇评论中也分享了类似的观点。

3. 最后，也是最为重要，却容易被忽视的一点是，GFW其实是在极力地弥补和掩盖其缺乏灵活性的根本性弱点。在这场猫与鼠的游戏中，审查者自知其在反应速度、灵活性上根本不是反审查社区的对手。 如果他们在敏感时期开始前的很长一段时间就开始部署他们新的秘密武器，那么反审查社区就有更多的时间去研究它，并发现绕过审查的新方法。到那时，如果他们不能灵活快速地改进自己的审查武器，那他们在政治敏感时期收紧网络管控的企图就会落空。David Fifield在这篇评论中也分享了类似的观点。

审查者有弱点

审查者缺乏灵活性是由其自身和其所面对的问题的本质所决定的。具体而言，其自身是庞大官僚机构的一部分，这不可避免的导致其内部运行效率低下，行为僵化。而其所面临的问题却又是在全国范围内大规模地监控和审查复杂多变的网络流量。可以想象，一款新型审查武器总是要经历预先研究，经费申请，立项，科研，产品开发，调试，真实流量调研，实验性部署，才能到最后的全国范围内的大规模部署和使用。其过程的漫长程度可想而知。

空口无凭。我们不妨把这次发布当作又一场观察（反）审查者反应速度的实验：看一看审查者要花多久才能封锁我们这次发布的这个有着诸多弱点的版本。

如何利用审查者的弱点？

针对审查者的弱点，我们想到了一些也许能让反审查工作在有限的资源下，事半功倍的原则：

1. 要对不完美的翻墙方案更宽容，不要过早放弃一个不完美的翻墙方案。我们前面说GFW灵活性不如反审查社区这个弱点通常被人们忽视，是因为看到许多的翻墙方案仅仅因为“存在弱点”就被过早的否定和扼杀了。这通常是因为反审查人员在设想自己是审查者时，更多的只考虑了第一步“预先研究”，觉得某个翻墙方案很容易被封锁，而忽视了真正的审查者所要面对的还有漫长的经费申请，立项，科研，产品开发，调试，真实流量调研，实验性部署，才能到最后的全国范围内的大规模部署和使用。事实上，如果我们的反审查人员花一个下午做好的反审查方案，在消耗了审查者半年的时间、精力、人力、物力、财力后被封锁了，其四两拨千斤的作用也早已经达到了。

2. 要增加翻墙方案的多样性，鼓励不同翻墙方案的百花齐放。在很多人的想象中，GFW因为拥有国家级的资源而成为一个完美的审查者，但实际上，有技术、能力、资源将上述审查武器研发流程一一走完的团队是有限的。我们反审查社区开发的翻墙方案越多种多样，这些有限的资源团队就越无法一网打尽所有的翻墙软件。而只要在某次大封锁中还剩下一种能用翻墙方案，信息就不至于完全阻塞。

3. 积极的报告新的审查事件，当新的审查事件发生时，迅速地测量新的审查机器，并将测量结果和绕过审查的方案分享给社区。达成这一项需要网民，研究者以及开发者三方一起沟通、努力、和配合。令人鼓舞的是，我们现在看到越来越多的人加入到这个协作之中，相互的配合也越来越默契顺畅。

4. 提前开发备用的翻墙方案。我们看到GFW为了弥补自身灵活性的不足而想到在政治敏感时期前几天才使用新的秘密武器，杀网民一个措手不及。那我们是不是也可以用相同的思路，在平时多开发几种备用的翻墙方案，然后像这次的发布一样，到了敏感时期之前才发出？这样，就算是审查者中的技术人员马上发现了新工具中的缺陷，知道了如何反制，也还是需要经过漫长的研发周期。

感谢

我们感谢David Fifield对文章初稿的反馈。

联系我们

正如前文所说的，我们发帖的目的就是想引起用户，研究人员和开发者们的讨论。因此我们欢迎您或公开地或私下地与我们分享您的使用体验或想法。我们私下的联系方式可见GFW Report的页脚。

概括:以量取胜，使用大量不同的协议来遏制gfw的服务器资源

Overview: Quantity wins, using a large number of different protocols to contain gfw's server resources

感谢！！不知对 TLS in TLS 的研究有任何结论或新的编译吗？

Thanks!!! I wonder if there are any conclusions or new compilations of the TLS in TLS research?

感谢！！不知对 TLS in TLS 的研究有任何结论或新的编译吗？
Thanks!!! I wonder if there are any conclusions or new compilations of the TLS in TLS research?

You may want to take a look at this post and this thread.

概括:以量取胜，使用大量不同的协议来遏制gfw的服务器资源

Overview: Quantity wins, using a large number of different protocols to contain gfw's server resources

是的，这样做本质还是捉迷藏，并不能长期逃避审查。毕竟 Shadowsocks 一直有 数据熵较高 的特征。

Yes, intrinsically this modification is a kind of hide-and-seek and it can't be used to gight against the censorship in long term. The Shadowsocks protocol has a feature known as high data entropy after all.

非常感谢，我使用这个方式在ipv6服务器上搭建完成，并可以使用。
只是在服务器端设置上，如果是纯ipv6服务器，需要将server设置改为：
"server": "::",

Thank you very much, I used this to build it on an ipv6 server and it works.
Just on the server side settings, if it is a pure ipv6 server, you need to change the server settings to
"server": "::",

不懂就问，基于游戏流量（https://github.com/MCModuleStudio/Minecraft-tunnel-server ）再加上加速器能否规避封杀

Ask if you don't understand, based on the game traffic (https://github.com/MCModuleStudio/Minecraft-tunnel-server ) plus accelerator can circumvent the blocking

Specifically, it is itself part of a large bureaucracy, which inevitably leads to inefficient internal operations and rigid behaviors.

I'd say it has been more efficient than before. It was a bureaucracy, but market forces have been introduced into the problem domain. Independent vendors compete to solve engineering challenges and optimize performance.

It is not hard to imagine that a new censorship weapon always has to go through the procedures of early-phase research, grants application, more formal scientific research, product development, debugging, surveying on real-world traffic, experimental deployment, and then the final nationwide large-scale deployment and use. The length of the process can take quite a long time.

Not always, probably not even close. The issue with the scientific workflow of censorship research was that the academic project lifecycle is very long and the deliverables are papers, not actual censorship systems and large scale results, and if there are any practical results, their ROIs are very unpredictable in both expenses and project schedule. That paradigm of censorship R&D being unsustainable doesn't mean there isn't another business mode that can make this work.

This is where I argue this area is now much more streamlined and efficient, because much of the censorship problems are no longer academic problems but industry engineering problems. The new variants of v2ray are not an interesting and novel academic research topic (blockchains, other new types of anonymous distributed networks may qualify for academic novelty). The variants are by nature not suited for the academic research workflow. It's entirely logical for these to be solved at the engineering level where a plethora of companies can compete to deal with the large diversity.

To compare the characteristics of the academic and industrial censorship R&D workflows:

• Academia: deals with a few very deep, very large, very complex systems, values novelty, cannot afford to handle large diversity which requires manpower (number of grad students is limited), delivers academic papers without guarantees for practical deployment or real world performance, project schedule is unpredictable, may fail and produce nothing.
• Industry: deals with a large quantity of simple, improvised, easy systems (easy as in usability), values real world performance, does not value theoretical novelty, can afford to handle diversity with manpower, delivers working systems with measurable real world performance, agile project schedules with possible expedited delivery, failure has severe consequences (bankruptcy or market irrelevance).

Why is machine learning very popular in this area of problem. It is because machine learning is actually more efficient than human learning at dealing with diverse protocols. With human learning it requires a large number of experts with deep understanding in networks and protocols but the time and budget to deal with a particular circumvention tool is unpredictable. With machine learning there is no requirement for deep domain expertise, and the cost for good performance is linearly or sublinearly correlated to the amount of raw compute power and ML system tuning manpower invested. This makes a very good deal for decisionmakers because ROIs are quite measurable and predictable in this case compared to the academic workflow.

Some relatively easy ideas I gathered to improve TLS in TLS tunnel survivability:

• Modify payload ClientHello padding to include more random sized bytes to confuse traffic analysis. This requires no change to the tunnel protocol. (via Large scale blocking of TLS-based censorship circumvention tools in China #129 (comment))
• Perform a real or realistic http request as the tunnel preamble to confuse traffic analysis systems that handles the starting part of a traffic flow (i.e. most of them). Issues: this adds connection latency, but can be amortized with multiplexing.
• Use multiplexing.
• Use simple random padding for the initial few round trips.
• Use a full-blown model-parroting traffic shaper to confuscate packet size and timing, but only for the initial few round trips (which can make implementation easier due to opportunities for hard coding). To make this happen, also:
  • A traffic model collection tool (candidate: chrome://net-export with a parsing script)
  • Traffic fingerprint self audit tools (proxy tools provide tooling to record traffic flow metadata and visualize it so users can share and compare the results with known traffic profiles, or centrally compared at some equivalents of tlsfingerprint.io).

概括:以量取胜，使用大量不同的协议来遏制gfw的服务器资源

Overview: Quantity wins, using a large number of different protocols to contain gfw's server resources

Wrong strategy. Then GFW enough to detect "valid" protocols and block all the others.

https://hostloc.com/thread-1082330-1-1.html
Here are some pictures that directly refer to GFW background analysis (maybe).
In the first picture, we can see that WS, Trojan is identified.
In the second picture, we can see the words naiveproxy, which can probably be identified accurately.
In the third picture, we can see that the A100 GPU is enabled at the top, and the website or app visited by the naiveproxy is directly analyzed.

After 3 hours of re-completing the manual,it still does not work,which is quite identical as before.

@database64128 also implemented another way to bypass the censorship. Because the protocol is changed, it requires updating both client and server to use it.

I wouldn't call it a protocol change, since anyone can easily implement the idea as an SIP003 plugin. This feature was added for convenience. It is not and probably will never be a part of the protocol.

Modify payload ClientHello padding to include more random sized bytes to confuse traffic analysis. This requires no change to the tunnel protocol.

I have recently made the change in database64128/shadowsocks-go@d98e94d to pad initial payload by default.

Use simple random padding for the initial few round trips.

Use a full-blown model-parroting traffic shaper to confuscate packet size and timing

To make these possible, payload chunks in Shadowsocks TCP need to support padding. I've made plans to add it for Shadowsocks 2023:

Shadowsocks 2022 and before:

 ---------------- 
|  length chunk  |
 ---------------- 
| u16 big-endian |
 ---------------- 

 --------------- 
| payload chunk |
 --------------- 
|   variable    |
 --------------- 

Shadowsocks 2023 (draft):

Length chunk:
 ---------------------- ---------------- 
| payload chunk length | payload length |
 ---------------------- ---------------- 
|    u16 big-endian    | u16 big-endian |
 ---------------------- ---------------- 

Payload chunk:
 ---------- ---------- 
| payload  | padding  |
 ---------- ---------- 
| variable | variable |
 ---------- ---------- 

My apologies as I may have missed some discussion about this earlier, but I was under the impression that the recent blocking targets only TLS-based circumvention tools. Is shadowsocks also affected?
I've two Outline servers that my friends in mainland have been using over 6 months and both of them survived the recent blocking. Is this patch preemptive?

昨天晚上11:20按群友建议购买了2台lightsail，
在配置翻墙前测试了IP都是干净的，都正常，没有被墙。
第一台配置了上面的shadowsocks，端口20060，加密方式aes-256-gcm
第二台配置了上面的shadowsocks，端口8878，加密方式chacha20-ietf-poly1305

客户端是Windows虚拟机新安装的系统，
访问内容是telegram客户端和youtube，
第一台在10.16早上9:40被墙端口，使用流量4.61GB，换端口到50505后，10:14被墙IP，使用流量341MB
换到第二台使用，11:09被墙IP，使用流量455MB

Last night at 11:20 as suggested by the group purchased 2 lightsail.
In the configuration of the wall before the test IP are clean, are normal, not walled.
The first configured the above shadowsocks, port 20060, encryption method aes-256-gcm
The second configured the above shadowsocks, port 8878, encryption method chacha20-ietf-poly1305

The client is a new installation of the Windows virtual machine.
The access to the content is telegram client and youtube.
The first one was walled at 9:40am on 10.16, using 4.61GB of traffic, after changing the port to 50505, the IP was walled at 10:14, using 341MB of traffic.
After switching to the second one, at 11:09, the IP was walled, using 455MB of traffic.

我昨天中午部署的，使用流量1g左右，暂时没有被墙，但是感觉不怎么好用啊

I deployed it yesterday at noon, using about 1g of traffic, and it's not walled for now, but it doesn't feel like it's working very well

概括:以量取胜，使用大量不同的协议来遏制gfw的服务器资源
Overview: Quantity wins, using a large number of different protocols to contain gfw's server resources

Wrong strategy. Then GFW enough to detect "valid" protocols and block all the others.

That's whitelisting, not in the scope of discussion

So I will mark these screenshots as suspicious.

I'll take these as bug reports.

HezeCaoxian:

昨天晚上11:20按群友建议购买了2台lightsail，
在配置翻墙前测试了IP都是干净的，都正常，没有被墙。
第一台配置了上面的shadowsocks，端口20060，加密方式aes-256-gcm
第二台配置了上面的shadowsocks，端口8878，加密方式chacha20-ietf-poly1305

客户端是Windows虚拟机新安装的系统，
访问内容是telegram客户端和youtube，
第一台在10.16早上9:40被墙端口，使用流量4.61GB，换端口到50505后，10:14被墙IP，使用流量341MB
换到第二台使用，11:09被墙IP，使用流量455MB

非常感谢您的报告。这样的用户报告及时、准确、详实，帮我们排除了很多其他可能性。我们现在立刻开始调查。

Thank you very much for your report. User reports like this are timely, accurate, and detailed, and help us rule out many other possibilities. We are now starting our investigation immediately.

My apologies as I may have missed some discussion about this earlier, but I was under the impression that the recent blocking targets only TLS-based circumvention tools. Is shadowsocks also affected?

Good question! The GFW has been able to accurately detect and block Shadowsocks, VMess, and Obfs4, and many other fully-encrypted protocols since November, 2022.

There are at least two ways the GFW blocks these protocols: immediate dynamic blocking of ports and relatively long term static IP or port blocking.

We know dynamic blocking has only been applied to connections from China to the IP subsets of many popular data centers like Amazon Lightsail, Digital Ocean and Alibaba cloud. They selectively deployed this technique for probably two reasons:

1. The censor knows their detection algorithm has a relatively high false positive and is not willing to harm 0.6% non-circumvention traffic;
2. Having a tight control over these VPS providers' IP subnet is sufficient to disrupt most users' connections. This is due to the nature that most of the Chinese users can only rent from a few VPS providers that accept payment methods they have.

How do you know if a certain IP address is under dynamic blocking? Try sending some random payload from China to an open port of that IP address:

head -c 100 /dev/urandom | nc -vn $ip $port

If after executing the command several times, your connections got timeout for 120 or 180 seconds, then you can confirm that the IP you tested is within the GFW's dynamic blocking IP subnets.

I've two Outline servers that my friends in mainland have been using over 6 months and both of them survived the recent blocking.

Thanks for your report. Are your servers located within one of those big name VPS providers? Or are you hosting them at some residential or institutional IP addresses that often do not have circumvention servers hosted?

To make these possible, payload chunks in Shadowsocks TCP need to support padding.

@database64128 maybe you know already, but it is possible to implement padding in Shadowsocks AEAD with no protocol changes.

#101 (comment)

Something similar is possible with Shadowsocks AEAD ciphers using encryptions of empty plaintexts, even though Shadowsocks doesn't have explicit support for padding. I heard this idea from @fortuna. When you need a source of bytes for padding, you can encrypt a zero-length plaintext, which gives you 34 bytes that decrypt to nothing: 2 bytes encrypted length field, 16 bytes length field authentication tag, 0 bytes encrypted payload, 16 bytes payload authentication tag. You can concatenate as many of these empty ciphertexts as you need to satisfy the traffic schedule.

Let's say you want to add npadding bytes of padding to an existing current_packet before sending. You encrypt enough empty plaintexts to make at least npadding bytes and append the first npadding bytes to the packet. The excess bytes become the beginning of the next packet to send.

padding = ""
while len(padding) < npadding:
    padding  = shadowsocks_aead_encrypt("")
current_packet  = padding[:npadding]
next_packet = padding[npadding:]
send(current_packet)

Thanks for your report. Are your servers located within one of those big name VPS providers? Or are you hosting them at some residential or institutional IP addresses that often do not have circumvention servers hosted?

Thank you for the information. In fact, right after I posted yesterday they got IP blocked. I guess I run out of luck eventually. They were rented from profitserver.net and located in Netherlands and Russia, respectively.

I'll set up new servers following the instructions provided here. Is there any additional steps I should consider (e.g., collecting pcaps for suspicious probes) to help investigate how they do blocking?

@gfw-report Could you clarify what the change you made to shadowsocks is?

@wkrp Thank you for letting me know. This workaround doesn't sound very flexible. The padding length is limited to multiples of 34. And it takes 2 read() calls for the server to process each 34-byte padding. My implementation also explicitly rejects empty payload chunks, which I do not plan to change.

I've seen more than a few developers in the Shadowsocks community with the sentiment that we must avoid protocol changes at all costs, that protocol changes won't be accepted by downstream developers or users. If anything, my experience with Shadowsocks 2022 has taught me otherwise. People want new protocols to experiment with. People are concerned about the fact that we keep introducing hacks and workarounds to "improve" Shadowsocks AEAD. The end result is a fragmented, divided community, where new developers start their own small circles, instead of joining existing ones, even though we all share the same goal of circumventing censorship.

Sometimes protocol changes are necessary. At this point, I consider recommending Shadowsocks AEAD to users to be irresponsible, because of the known vulnerabilities. Like @gfw-report said, it's a cat-and-mouse game. The protocol needs to rapidly iterate when faced with new challenges.

We are now starting our investigation immediately.

@gfw-report In case you didn't already know and it might be helpful to your investigation, shadowsocks-rust by default only enables Shadowsocks 2022's replay protection. The bloom filter for older AEAD and stream ciphers is disabled by default.

diwenx:

I'll set up new servers following the instructions provided here. Is there any additional steps I should consider (e.g., collecting pcaps for suspicious probes) to help investigate how they do blocking?

Yes, that would be very helpful because the more eyes we have, the more likely and quickly we spot any new censorship actions.

database64128:

shadowsocks-rust by default only enables Shadowsocks 2022's replay protection. The bloom filter for older AEAD and stream ciphers is disabled by default.

Thanks for the heads up. As soon as we support this encoding scheme on Shadowsocks 2022, we can start recommending users to use Shadowsocks-2022. Any help to implement it is more than welcome!

BTW, in case you are concerned about the active probing vulnerability, this modified Shadowsocks has been proved to be able to evade the current active probings because the GFW doesn't suspect its traffic in the first place.

This workaround doesn't sound very flexible. The padding length is limited to multiples of 34.

I can respect your development decisions, but I believe you are mistaken on this point. There is no requirement that TCP segment boundaries be aligned with AEAD ciphertext boundaries. For example, if you need 75 bytes of padding for the current segment, you can generate 102 bytes and only use the first 75 bytes of it for the current segment. The remaining 27 bytes become the beginning of the next segment. (In general, you keep a buffer of available data. If you want to send a segment that's smaller than the amount of data in the buffer, you break off the head of the buffer and only send that. If you want to send a segment that's larger than the buffer, you increase the size of the buffer with padding ciphertexts until it is sufficiently large.)

You are correct, however, that the receiver can only consume the padding 34 bytes at a time, which for large amounts of padding means many recv system calls and verify decrypt operations.

For example, if you need 75 bytes of padding for the current segment, you can generate 102 bytes and only use the first 75 bytes of it for the current segment. The remaining 27 bytes become the beginning of the next segment.

Unless the proxy program runs its own TCP stack (which could create its own problems (e.g. fingerprinting of the TCP stack)), I don't think it's realistic to control TCP segment boundaries during mass payload transfers.

I can respect your development decisions, but I believe you are mistaken on this point.

I think you misunderstood what I meant. My understanding is that the proxy program might want to add padding between individual payload chunks to change the characteristics of outgoing TCP segment sizes. And that works best when you can add as many bytes of padding as you want. Your proposed method of doing smaller writes cannot guarantee smaller TCP segments, even if you enable TCP_NODELAY.

Can you add support for udp, including uot
能否增加对udp的支持，包括uot

Thank you guys for your help

When using proxies such as shadowsocks or v2ray, my upload speed is very very bad. example my download speed is at 1 mb/s, my upload speed goes down to 30 kb/s
Are there ways to solve this issue?

When using proxies such as shadowsocks or v2ray, my upload speed is very very bad. example my download speed is at 1 mb/s, my upload speed goes down to 30 kb/s Are there ways to solve this issue?

There's nothing in the protocol itself that would make it that slow. You would have to do A/B testing to narrow down the cause of the problem, e.g. overloaded server, congested route, deliberate throttling by ISP, or whatever.

Is there a specific reason that you use "pure" ShadowSocks (without plugins such as V2Ray, Xray, etc)? @gfw-report

If you are a V2Ray user, you can turn on the ExperimentReducedIvHeadEntropy option to avoid the GFW's detection and blocking. The advantage of this option is that you do not need to make any changes to your servers.

Hello there, currently I am using wss (vmess tls nginx), I use LEMP, then I use certbot to issue ssl, finally I add wordpress.
I am wondering how to add this ExperimentReducedIvHeadEntropy option?

for clients I use,
windows: Clash for Windows;
android: v2rayNG, Clash for Android;
macOS: clashX;
iOS: shadowrocket.

what I learnt and found out mostly close tothe description are located here in v2fly.org (https://www.v2fly.org/config/protocols/vmess.html#userobject), then search "experiments: string".

So I took clash for windows as an example, and I add this in clash for windows yaml file:

  # 仅适用于设置 allow-lan 为 true 时
  # 192.168.122.11: 绑定单个 IPv4 地址
  # "[aaaa::a8aa:ff:fe09:57d8]": 绑定单个 IPv6 地址
  # bind-address: "*", "*": 绑定所有 IP 地址
  experimental:
    ignore-resolve-fail: true # 忽略 DNS 解析失败，默认值为true
    Experiment-Reduced-Iv-Head-Entropy: true # reduce entropy
    experiment-reduced-iv-head-entropy: true # reduce entropy
    ExperimentReducedIvHeadEntropy: true # reduce entropy
    experimentreducedivheadentropy: true # reduce entropy

and then at the server side I add this in ClientObject (https://www.v2fly.org/config/protocols/vmess.html#clientobject) under InboundConfigurationObject:

  {
    "email": "[email protected]",
    "id": "random uuid",
    "level": 0,
    "alterId": 0,
    "experiments": "ExperimentReducedIvHeadEntropy"
  },

Finally I also add maxEarlyData (https://www.v2fly.org/config/transport/websocket.html#websocketobject) in WebSocketObject.

  "wsSettings": {
    "path": "/x",
    "headers": {
      "Host": "example.com"
    },
    "maxEarlyData": 1024,
    "earlyDataHeaderName": "Sec-WebSocket-Protocol"
  },

I know these are probably wrong but all these echos back to the question, how to add 'ExperimentReducedIvHeadEntropy'. Although my servers get blocked daily but I think wss is still a very good method. Thank you in advance.

The Outline Client now supports a prefix parameter that gets used as the first bytes of the connection salt to disguise it as an allowed protocol. The prefix is specified on the Access Key. Usage details and examples can be found at https://www.reddit.com/r/outlinevpn/wiki/index/prefixing/

Combined with Dynamic Access Keys, you can use different prefixes for different sessions.

One advantage of our approach is that downloading the client or inspecting its code doesn't give the censor knowledge to know what to block. Furthermore, it enables service providers to change prefixes easily and move faster than the censor.

The Outline Client now supports a prefix parameter that gets used as the first bytes of the connection salt to disguise it as an allowed protocol. The prefix is specified on the Access Key. Usage details and examples can be found at https://www.reddit.com/r/outlinevpn/wiki/index/prefixing/

Combined with Dynamic Access Keys, you can use different prefixes for different sessions.

One advantage of our approach is that downloading the client or inspecting its code doesn't give the censor knowledge to know what to block. Furthermore, it enables service providers to change prefixes easily and move faster than the censor.

FYI: Tests have been done, users report that it does not working in Iran even with these modifications. However, it might be used in China.

@gfw-report unfortunately doesn't work for Iran.

I just read the paper How the Great Firewall of China Detects and Blocks Fully Encrypted Traffic, which seems to be the research related to this modification. However, I also noticed that this research in 2021 has a contradictory conclusion regarding entropy. Did the blocking strategy change during the 1 year period?

@gfw-report
How do you know if a certain IP address is under dynamic blocking? Try sending some random payload from China to an open port of that IP address:

head -c 100 /dev/urandom | nc -vn $ip $port

If after executing the command several times, your connections got timeout for 120 or 180 seconds, then you can confirm that the IP you tested is within the GFW's dynamic blocking IP subnets.

Sorry for necro but I did the test yesterday on a proxy server that I set up for my friend (only one user) and there was no timeout after reruning the command 200 time while the proxy server was blocked last year and unblocked few months later. It's an AWS Tokyo EC2 instance so I assume both the IP and ASN are red flags according to the latest research.

On the server end, I ran
nc -l -p 10000 -k -v
and my friend ran your command with for loop for 200 times.

Does it mean the block last year was just a false positive ? or did the GFW upgrade the algorithm?

BTW, do you have any insight of IPv6 traffic? According to some, IPv6 traffic is affected by neither active probing and passive analysis.

Hi @KaraRyougi,

I just read the paper How the Great Firewall of China Detects and Blocks Fully Encrypted Traffic, which seems to be the research related to this modification.

Thank you very much for your interest in the paper. And yes, you are right that this is the "modified Shadowsocks" we mentioned in the paper: https://gfw.report/publications/usenixsecurity23/en/#sec:responsible-disclosure

However, I also noticed that this research in 2021 has a contradictory conclusion regarding entropy. Did the blocking strategy change during the 1 year period?

Great question! Thank you for giving us the opportunity to clarify on that.

We did not observe any changes of the blocking strategy from November 2021 to the ceased blocking in March 2023. The contradicting statement was simply due to our lack of understanding of the system in 2021.

In particular, at that time in 2021, we did not fully understand the GFW's new censorship system and thus made the false/inaccurate statement in the preliminary report that "[b]locking is not based on entropy.... we find that entropy does not tell the whole story. Some payloads with very high entropy never trigger blocking, whereas other payloads with very low entropy can trigger blocking immediately".

We confirm that the GFW was indeed using the monobit testing as a measurement of randomness/entropy.

Hi @yoshyv,

We really appreciate your spirit of measuring the censorship system yourselves.

I did the test yesterday on a proxy server that I set up for my friend (only one user) and there was no timeout after reruning the command 200 time while the proxy server was blocked last year and unblocked few months later. It's an AWS Tokyo EC2 instance so I assume both the IP and ASN are red flags according to the latest research.
...
Does it mean the block last year was just a false positive ? or did the GFW upgrade the algorithm?

The reason why you couldn't observe the blocking anymore is likely because the GFW has stopped dynamically blocking fully encrypted traffic since sometime between Tuesday March 7, 2023 and Wednesday March 15, 2023.

It may be because the politically sensitive time has passed and the censor is thus less willing to tolerate the collateral damage. Specifically, there were two politically sensitive conferences in China ended on Monday March 13, 2023:

• https://en.wikipedia.org/wiki/2023_National_People's_Congress
• https://zh.wikipedia.org/wiki/中国人民政治协商会议第十四届全国委员会#第一次会议

We documented this observation in this directory and README in the repo: https://github.com/gfw-report/usenixsecurity23-artifact/tree/master/artifacts/ceased-dynamic-blocking

BTW, do you have any insight of IPv6 traffic? According to some, IPv6 traffic is affected by neither active probing and passive analysis.

Great question. We do not have an answer to this question because we haven't done any testing over IPv6 yet.

So from my understanding, the current investigation is only focusing on "real-time TCP block", which seems to be only part of how GFW works now. I suspect there are also offline detections. Evidence includes the reports of blocking in this thread, and other reports of UDP-based protocols such as Hysteria being blocked.

@gfw-report
We documented this observation in this directory and README in the repo: https://github.com/gfw-report/usenixsecurity23-artifact/tree/master/artifacts/ceased-dynamic-blocking

Thanks again for the heads up.

Are the affected AS and IP prefix the same for static blocking and dynamic blocking? Have you collected the data in this regard?
Since there's no way to find out the affected IPs in a short period of time via dynamic blocking test now, is there any other method to find out which IPs are more likely to be blocked?

Is the original shadowsocks (2017) still considered a viable protocol to bypass GFW now? How to take proactive approach to prevent proxy server being IP address blocked? In fear of IP blocking, I haven't used my residential network to set up shadowsocks proxy as my IP address is not dynamic and doesn't change over years.

每年花50-200美元租【亚马逊/微软/Oracle/阿里/百度/腾讯】在【新加坡/北美】的VPS作为代理，随便用个什么加密协议只要数据不被破解，即可堂堂正正访问外网资源，速度极好，还不会被封。实在太担心，再加点洋葱佐料。简单、安全。

Spend $50-200 per year to rent [Amazon / Microsoft / Oracle / Ali / Baidu / Tencent] in [Singapore / North America] VPS as a proxy, casually use what encryption protocols as long as the data is not cracked, you can squarely access to the outside resources, the speed is excellent, but also will not be blocked. If you're too worried, add some onion spices. Simple and safe.