libtorrent有无考虑开发一下基于UTP的UDP NAT1打洞 #6331
Comments
Unless "NAT1 hole punching" is a term of art describing some specific kind of hole punching, it already is. It's limited to full cone NATs though. |
|
我使用qbittorrent测试，并不支持nat1打洞 translate |
|
so, is this a bug report? Can you provide more details of your test case, such as logs from all 3 peers (the two NATed as well as the rendezvous peer) and the kind of NATs the peers are behind. Perhaps some independent test report showing that the NATs are indeed full cone (or at least one of them). |
|
also, you keep saying "nat1" is the different from just "NAT"? |
NAT大致分为下面四类
我们看看不同NAT之间的NAT打洞。NAT打洞需要Server配合，需要2种Server：
维基百科：https://zh.wikipedia.org/wiki/%E7%BD%91%E7%BB%9C%E5%9C%B0%E5%9D%80%E8%BD%AC%E6%8D%A2 translate
Let's look at NAT hole punching between different NATs. NAT hole punching requires the cooperation of the server, and two types of servers are required:
Wikipedia: https://zh.wikipedia.org/wiki/%E7%BD%91%E7%BB%9C%E5%9C%B0%E5%9D%80%E8%BD%AC%E6%8D%A2 |
如果libtorrent以前支持nat，那则是一个bug报告 translate |
|
Ok, so when you say "NAT1", you mean "full cone NAT". So when you say just "NAT", do you mean all kinds of NATs? if so, libtorrent (nor uTorrent) has ever supported "NAT", but supports "NAT1". a youtube link is not an effective medium to provide details on what's happening in libtorrent. |
|
I suspect that you are unsure under what scenarios hole-punching is expected to work. Let's start there. What scenario are you testing? See my questions in #6331 (comment) |
如果你对这个 issues 感兴趣的话，可以根据我提供的测试方法测试一下，我不是开发人员不太知道如何描述，测试场景在上方有描述过 translate |
|
需要设备C能连接上设备B translate |
@1265578519 What you call UDP hole-punching here is more precisely the ICE connection negotiation technique, as used by WebRTC. If this is what you want, you can enable WebTorrent support in libtorrent (WebTorrent is basically BitTorrent over WebRTC Data Channels).
It's not meaningless, it's basically a simpler implementation of a UDP hole-punching process similar to ICE. The relaying peer acts as both the STUN server (to get the external endpoint after NAT mapping) and the signaling server (to make the other peer connect at the same time to overcome NAT filtering).
Note those categories should be considered legacy terminology as they are confusing and do not really reflect real implementations. The IETF clarified this with new terminology in RFC 4787. See https://en.wikipedia.org/wiki/Network_address_translation#Methods_of_translation :
With this terminology, UDP hole-punching techniques require at least one of the two NATs to feature Endpoint-Independent Mapping. |
|
感谢楼上回复issues，具体怎么实现NAT1打洞就要看各位开发者提交pr。 translate |
|
My point is that UDP hole-punching is already implemented: BEP55 implements it already, and if you want ICE specifically, WebTorrent, which you can activate in the settings, implements it already. If it doesn't work with you setup, it's most probably a problem with the setup itself, or a bug with libtorrent in a pinch. (Since you said "there is not any software on the market that does hole punching", it's probably a problem with your setup or your understanding of hole-punching.) Like said before in #6331 (comment), you need to provide more information, like logs. A wild guess is that peer A has only the TCP port open, and therefore can't be used as relaying peer for hole-punching (You need a least one already-connected uTP peer for BEP55 hole-punching). |
|
我需要的不是WebTorrent，而是基于NAT1的打洞 translate |
|
I watched the youtube vid OP provided, and basically all OP did is that:
One interested thing is that his host machine is only connecting to the seeding machine with TCP/BT. In OP's vid the host machine tried to connect to the NATed peer with only TCP connection, too, which is guaranteed to fail in this case. But anyway, he didn't provide further detailed information like "what peer info did the host machine get", "which method / protocol did the host machine tried to use to connect to the NATed peer", and "what kind of failure the connection from his host machine to the FullCone one ended up with". Another thing is that OP's host machine is located in China mainland, (actually OP should try to disguise your IP with VPN / proxy or people will find out that this machine is probably located at someplace near Times Garden Kweilin.) and as far as I know, ISPs at China mainland restrict UDP connection so hard that you can't really use UDP to connect / transfer sh*t abroad, sometimes not even usable with another peer inside China mainland as well. I tried similar things using qTox with my friend while we are in the same province and the UDP connection is pretty f*cked up. I also tried restricting qBittorrent to only allow μTP / UDP to seed 2 TB in demand files consisted of over 120 hot torrents, and can only get about ten active UDP connected peers, let along five of them reside in China, while when I use TCP_bbr, I can seed to over 154 peers at full upload bandwidth. I think OP should use another abroad VPS or a proxied one to replace the machine in China in this test. |
|
感谢回复跟进issues，目前比特彗星1.81版本是全世界第一款支持NAT1 UDP打洞的软件，可以体验 Thanks for replying to follow up issues. At present, BitComet 1.81 is the first software in the world to support NAT1 UDP hole punching. You can experience it 一些相关使用方法 检测自身网络是NAT几 公网IP绿灯下是否还需要进行UDP打洞 UTP和UDP有什么关系 为什么启用UTP后连接成功后传输速度这么慢 其它BT软件支持UDP打洞吗 如何启用该版本的UDP打洞功能 Some related usage methods Check if your own network is NAT Do you still need to perform UDP hole punching under the green light of the public network IP? What is the relationship between UTP and UDP Why is the transmission speed so slow after a successful connection after UTP is enabled Does other BT software support UDP hole punching? How to enable this version of UDP hole punching |
|
I'll be a little bit disappointed but also amused if all you prepared is only for self advertising your "mod" to a closed source software. |
|
所以我才来libtorrent发issues，希望开源软件能够支持NAT1 UDP打洞，只要libtorrent能够支持，qbittorrent就可以更新libtorrent实现支持 That’s why I came to libtorrent to post issues, hoping that the open source software can support NAT1 UDP hole punching |
Please do help as you claimed. |
|
在我发这个issues之前，bitcomet并不支持NAT1打洞，他们是2021年10月1日更新支持，我是在2021年7月23日发的issues Before I posted this issue, bitcomet did not support NAT1 hole punching. They updated support on October 1, 2021. I posted the issue on July 23, 2021. |
我在上面描述的非常清楚了，你可以看一下之前的issues信息 What I described above is very clear, you can look at the previous issues information |
|
我就是因为仔细看了所有前文才如此不适，你应该提供 @arvidn 需要的日志和详细重现步骤作为你非要断定 libtorrent 有漏洞的根据。你没有提供日志的问题开发者已经提过了，你的测试步骤需要拓展我也已经提过了。 I feel so uncomfortable just because I've read all the previous posts carefully. You are the one who should provide the logs and detailed reproduction steps that @arvidn requested as the basis for your conclusion that libtorrent is bugged. The problem that you didn't provide logs has already been mentioned by the developers. Also, I've already suggested that your test steps might need to be expanded as well. |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
目前libtorrent并未支持nat1打洞，已经被机器人标记过时关闭，估计以后也不会支持了。。毕竟开发难度巨大 |
|
Google Translate of previous post: ...Has already been proven to be incorrect. Until the poster does the work required by: |
|
https://www.youtube.com/watch?v=wOLp7f-gb1M |
|
Are you joking? |
I'm afraid not. If you don't perform hole puncturing, you won't even have a port because you're only listening on your router, not the ISP's NAT device. You may want to read this: |
Sorry, but your understanding is actually the wrong one. Once your p2p client's NAT status is determined, it includes your Carrier Grade NAT restriction as well. So if you actually did a NAT behavior test and got so called "NAT1", you can just keep your external endpoint alive and just report it to the tracker. By the definition of so called "NAT1", other clients will be able to communicate with you by talking to that endpoint, and it won't change too easily. Nat FullCone or NAT1 is not the modern terminology for describing these behaviors anymore. Should be "Endpoint-Independent NAT mapping", see RFC-4878 or Tailscale's blog for an easier explanation. But since you've written "not the ISP's NAT device", I'd suggest you to first do research about how to test your NAT behavior, and how can a STUN server help you with that. |
Thanks for your reply, I totally understand what you mean. My reply just now was to explain to k79e that hole punching (which in my understanding includes communicating with the STUN server) is necessary. Without this process, the ISP's NAT device would not be aware of the existence of this service. I haven't looked into libtorrent's support for this, but it would obviously require some extra work rather than being as easy as k79e describes
|
Well, for the case of bittorrent clients, they actually don't communicate to STUN servers. They fetch a list of living peers from dht or pex or tracker, then use other globally routable (or maybe Endpoint-Independet-Mapping) peers as a STUN server equivalent, which is already implemented and mentioned previously. See BEP55 |
|
截止2023-11-14, libtorrent 2.0.9依旧不支持使用UDP的NAT1打洞 As of 2023-11-14, libtorrent 2.0.9 still does not support UDP NAT1 hole punching |
Thanks for the wonderful explanation! By the way, considering the restrictions you mentioned on UDP by ISPs in mainland China, is it possible to implement a TCP hole punching function similar to natter in libtorrent #7207? Currently, we need to use a script like this. |
|
TCP hole punching is way more difficult to be implemented correctly. And because BitTorrent is a protocol which needs all (different brands of) clients to support it for that feature to function as expected, I highly doubt whether if it will be accepted and popularized even if you created a perfect draft BEP and submitted it. With that said, as HTTP3 uses QUIC thus UDP right now, I doubt if you will still experience severe degrade of UDP connection quality. What's more is that, most main stream ISP in China now support ipv6 fully. So it's not hard at all now for normal users in China to obtain a globally routable IPv6 address and setup correct firewall rules to allow incoming connection. It's already a consensus (from even IPFilter's devs years ago) that there shouldn't be more resource put into NAT workarounds. Even then, most of the time when I know some Chinese users who don't have a usable Endpoint-Independent-Mapping environment, it's because of bad router / modem settings and port forwarding / UPnP settings. The three mainstream ISP in China only have CGNAT for IPv4 and it's doing Endpoint-Independent-Mapping already. |
Would you care to define "UDP NAT1 hole punching"? |
Understanding Full Cone CGNATIf a peer is behind a full cone CGNAT, each of its communications will be assigned a shorter lease external port that does not match its listening local port. If the peer reports its local port to the tracker and other peers, the connection will fail to establish. Requirements for Successful ConnectionsA peer behind a full cone NAT must have a way to:
This process is referred to as hole punching. Current Limitations of qbittorrentBased on my experience, qbittorrent (which is based on libtorrent) does not currently support hole punching for full cone NAT. It reports the incorrect local listening port to the tracker instead of the external mapped port. Solutions Found by Users in ChinaUsers in China have discovered methods to address this issue, such as:
This way, qbittorrent will report the correct port number C to the tracker. Additional Tools and ScriptsI noticed that qbittorrent supports UPnP/NAT-PMP protocols. In response, I wrote a Python script that runs on the gateway device to:
The script is available at: https://github.com/faithleysath/natpmp-stun. Invitation for DiscussionIf anyone has more questions, please reply to let me know. I am happy to discuss. 理解全锥CGNAT如果一个节点位于全锥型CGNAT后面，每次通信都会被分配一个与其本地监听端口不匹配的短期租约外部端口。如果节点将其本地端口报告给跟踪器和其他节点，连接将无法建立。 成功连接的要求位于全锥NAT后的节点必须能够：
这个过程称为打洞。 qbittorrent当前的限制根据我的经验，qbittorrent（基于libtorrent）当前不支持全锥NAT的打洞。它向跟踪器报告了错误的本地监听端口，而不是映射的外部端口。 中国用户找到的解决方案中国用户发现了解决此问题的方法，例如：
这样，qbittorrent将向跟踪器报告正确的端口号C。 其他工具和脚本我注意到qbittorrent支持UPnP/NAT-PMP协议。对此，我编写了一个运行在网关设备上的Python脚本：
脚本可在此处获得：https://github.com/faithleysath/natpmp-stun。 讨论邀请如果有人有更多问题，请回复让我知道。我很乐意讨论。 |
|
比特彗星使用UTP协议进行NAT1 UDP打洞实现原理 比特彗星NAT1打洞主要为以下两个方面 对于DHT 回复包 对方peer获得端口号与NAT1端口一致 对于PEX 比特彗星的打洞和tracker汇报时发送为监听端口，而不是NAT端口，当前版本没有借助tracker去打洞 PT打洞原理 截至目前libtorrent-2.0.10版本，依旧无法进行NAT1打洞 借助其它脚本实现外部方式的打洞也是可以的，但是实现过程比较复杂，不如直接在软件中内置PEX、DHT打洞算法，毕竟讲究的是默认值，不可能人人都会怎么去使用脚本 |
Although libtorrent still hasn't implemented what you expected, with the widespread adoption of IPv6 in China and the strict regulation of v4 NAT, full cone NAT (which you referred to as NAT1) is becoming increasingly irrelevant. |
|
行了行了 真想要的话自己动手吧. 找些pex方面的信息, 留结后人. udp tracker忽略一切报文里面的port, 只用连接信息里面的port, 这样一定不会错. 还不需要client修改. dht和pex就不清楚. 没见过文档. 有信息谁都能设计出方案. dht还是老了, 要是能加个显式nat tag, 也不会闹的错的信息满天飞. |
|
天哪只要ut_holepunch的logic修一下就能支持nat1. If the relaying peer is connected to the target peer, and the target peer supports this extension, the relaying peer sends a connect message to both the initiating peer and the target peer, each containing the endpoint of the other. 这都是什么事哇, 跟本不需要修改. 都不需要新加元素, 己有的就够用了. 但是就是有问题. 有人作准备么? 连接头信息叫什么? 是那个文件里的? 己有打洞程序位置是哪里? |
|
多半美国佬人人都是公网ip，没有体验过nat1 |
|
@1265578519 他们一开始不知道谁写的logic, 这个200x年就不应该是问题. 但是没人测试才这样!!!!! |
Actually, the author is a Swedish guy, not American, haha. But you're right, the author probably didn't have the chance to test this situation; they can't imagine how strict the internet in China is. 实际上作者是瑞典老哥，哈哈 不过你说的对，作者大概率没有办法测试这种情况，他们无法想象中国的网络有多严格。 |
libtorrent有无考虑开发一下基于UTP的UDP NAT1打洞
比特彗星那边也在做了，不过开发进度有点慢，，而且做出来的效果不尽人意，还在等待后续更新
https://www.cometbbs.com/t/178%E6%B5%8B%E8%AF%95%E7%89%88/59260
以前我也测过utorrent，是根本无法NAT打洞的。。。包括 qBittorrent 也不行，可能开发者都搞错了，现在市面上所有的BT软件，，基本都是需要对方为外部网络，TCP端口开放，然后通过升级到UTP传输而已，而不是打洞
这就是我上面说的，其实目前市面上没有任何一款软件做了打洞，单纯的只是升级为同端口的udp传输而已。
比特彗星那边正在实行的就是基于BEP55来实现PEX报文传递的是客户端的监听端口，这种打洞是没有意义的
测试NAT1打洞需要至少3台设备，测试环境如下
做种服务器A
防火墙开放TCP端口，做种客户端设置为UTP启用
下载服务器B
B服务器在路由器中封堵TCP端口，NAT值为1
一样设置为UTP启用，执行下载，此时服务器A可以看到一个来源为被动的服务器B请求。
下载服务器C
C服务器在路由器中封堵TCP端口，NAT值为4
设置UTP启用，执行下载，测试是否与服务器B连接成功
服务器A和服务器B可为公网IP的云VPS，服务器B在系统防火墙高级设置封堵TCP入站即可，服务器C可为家中电脑设备
顺便吐槽一下，，qBittorrent 这么多年了，单线程传输速率还是这么烂，5MB/S极限了，比特彗星可以跑到80MB/S
https://www.youtube.com/watch?v=pO_nIMSKWd8
translate
Is libtorrent considering developing UTP-based UDP NAT1 hole punching?
Bit Comet is also doing it there, but the development progress is a bit slow, and the results are not satisfactory, and we are still waiting for follow-up updates.
https://www.cometbbs.com/t/178%E6%B5%8B%E8%AF%95%E7%89%88/59260
I have tested utorrent before, and it is impossible to make NAT holes at all. . . Including qBittorrent also does not work, maybe the developers are mistaken, all BT software on the market now basically requires the other party to be an external network, open the TCP port, and then upgrade to UTP for transmission, instead of making holes
This is what I said above. Actually, there is not any software on the market that does hole punching. It is simply upgraded to udp transmission on the same port.
What Bit Comet is implementing is based on BEP55 to realize the PEX message transmission is the listening port of the client. This kind of hole punching is meaningless.
At least 3 devices are required to test NAT1 hole punching. The test environment is as follows
Seed server A
The firewall opens the TCP port, and the client is set to UTP to enable
Download server B
Server B blocks the TCP port in the router, and the NAT value is 1
The same is set to enable UTP, and download is performed. At this time, server A can see a request from server B whose source is passive.
Download server C
Server C blocks the TCP port in the router, and the NAT value is 4
Set UTP to enable, execute download, and test whether the connection with server B is successful
Server A and Server B can be cloud VPS with public IP, server B can block TCP inbound in the advanced settings of the system firewall, and server C can be a home computer device
By the way, qBittorrent has been around for so many years, the single-threaded transmission rate is still so bad, 5MB/S is the limit, BitComet can run to 80MB/S
https://www.youtube.com/watch?v=pO_nIMSKWd8
The text was updated successfully, but these errors were encountered: