Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support proxy network #8

Closed
hanguokai opened this issue Jan 16, 2018 · 34 comments
Closed

Support proxy network #8

hanguokai opened this issue Jan 16, 2018 · 34 comments

Comments

@hanguokai
Copy link

To use proxy network, like http(s) or socks5, I use http_proxy and https_proxy environment variable that are supported by many tools. Currently, clasp can't work in proxy environment, for example clasp login and clasp clone.

@grant
Copy link
Contributor

grant commented Jan 16, 2018

I'm not sure what this would change would entail.
Can you be more specific about repro steps, some links, and a more detailed proposal on what commands this would be needed for?

It looks like this request would want to add something like:
--proxy "socks5://127.0.0.1/"
to every command.

Similar issue: ytdl-org/youtube-dl#402 (comment)

@hanguokai
Copy link
Author

Yes, clasp need to add --proxy option, like youtube-dl, curl, gcloud, npm, git, etc.

http_proxy environment variable is the same meaning as --proxy, but detect by command if no --proxy option specified.

I am using socks5 protocol, there are others like http, https.

@grant
Copy link
Contributor

grant commented Feb 20, 2018

@hanguokai Can you add proxy rule to the URL googleapis.com so that every googleapis.com request is proxied?

@grant
Copy link
Contributor

grant commented Mar 6, 2018

Pull requests are welcome.
I'm closing this since I don't see much demand for this feature. Will reopen if there's demand or more discussion.

@grant grant closed this as completed Mar 6, 2018
@HeavensBlade
Copy link

I would also like to see a proxy option.
Since I'm developing behind a corporate proxy I always need to activate my smartphone hotspot.

@nepsilon
Copy link

I'm using clasp from a censored internet where all things Google are blocked and need the proxy feature. Using proxy environment variables doesn't work.

@grant
Copy link
Contributor

grant commented Apr 27, 2018

Why doesn't using proxy environment variables not work? Every request goes through http://googleapis.com/.

Does every application you use require a proxy feature?

@hanguokai
Copy link
Author

Aha, please look at the same issue for firebase command line tool firebase/firebase-tools#155 . Nearly 2 years and 81 comments! Under certain conditions, people can't connect internet directly. That's it.

@nepsilon
Copy link

@grant Yes, all blocked domains and IP requires a VPN or proxy. It's very standard in places like China.

@grant
Copy link
Contributor

grant commented Apr 27, 2018

Unfortunately G Suite and many of Google's services are blocked in China. I can't solve that issue. I'm guessing the Firebase tool would probably prefer to not be blocked too to solve that linked issue.

I don't understand why environment variables don't work, do you really need every single tool you use to implement a custom --proxy option?

Underneath it all, we use googleapis. It supports a proxy (HTTP_PROXY, HTTPS_PROXY). Does that not work?

To implement this feature, I'd need:

  • A reasonable number of people to request and use the proxy feature (👍 the issue)
  • A design doc for this feature.
  • Ideally a pull request with an initial implementation.

To be honest, I have no use for this feature and still don't understand why you can't use a proxy elsewhere in the tech stack.

@valen22br
Copy link
Contributor

valen22br commented Aug 20, 2018

Hello, I'm running through the same problem. I tried setting up my user variables with proxy settings, and I'm still receiving the error below:
Error retrieving access token: Error: connect ETIMEDOUT 172.217.18.170:443

Is there a workaround for this issue?

@grant
Copy link
Contributor

grant commented Aug 20, 2018

I'm in favor of adding a proxy option, but to be honest, I don't know how to run a proxy and I don't know why env variables don't work like in the googleapis README:

export HTTP_PROXY=http://localhost:3128
export http_proxy=http://localhost:3128
export https_proxy=http://localhost:3128
export HTTPS_PROXY=http://localhost:3128

I did a bit more research.

  • If someone wants to try this, add a proxy variable to this line of code: https://github.com/google/clasp/blob/master/src/auth.ts#L40
    Then rebuild clasp and test it out, and it works, that'd be awesome. Then I can update clasp to include this proxy param.
  • If someone (@hanguokai, @nepsilon) describes exact steps on how to download a proxy, see the error in clasp using some tool, I'll try to reproduce the problem and create a solution.
  • If you need help with a pull request, or building clasp, LMK too.

This is the most upvoted issue, and it looks like user variables doesn't work, so I'm in favor of support this feature in clasp, but I'll need some help for this one.

@grant grant reopened this Aug 20, 2018
@valen22br
Copy link
Contributor

valen22br commented Aug 21, 2018 via email

@valen22br
Copy link
Contributor

valen22br commented Aug 21, 2018 via email

@grant
Copy link
Contributor

grant commented Aug 21, 2018

@valen22br Follow these instructions https://github.com/google/clasp#develop
Let me know if you can't build clasp.

@valen22br
Copy link
Contributor

@grant the instructions worked for me. I will start playing with the proxy settings and will let you know the results.

@hanguokai
Copy link
Author

hanguokai commented Aug 21, 2018

I did some research about google-api-nodejs-client. It says it support proxy.

But in fact, googleapis use axios as it's http client library that doesn't support socks proxies. So export https_proxy=socks5://127.0.0.1:1080 or setting (socks) proxy option in code doesn't work.

In theory, googleapis should support http/https proxy like export http_proxy=http://localhost:3128. I didn't research whether there is a problem with this method.

PS: I know gcloud command support proxy including socks proxy. And firebase command had problem for supporting proxy, but recently it seems support http proxy(not socks proxy) in 4.1.2.

@grant
Copy link
Contributor

grant commented Aug 21, 2018

I don't know the difference between socks proxies and non-socks proxies. Will a non-socks proxy work?

@valen22br
Copy link
Contributor

I don't know either.
From my tests I got the following results:

  1. No HTTP_PROXY or HTTPS_PROXY environment setup. In this scenario, the error was: Error retrieving access token: Error: connect ETIMEDOUT 172.217.18.170:443
  2. With HTTP_PROXY or HTTPS_PROXY environment setup. Now I got another error: Error retrieving access token: Error: write EPROTO 101057795:error:140770FC:SSL
    routines:SSL23_GET_SERVER_HELLO:unknown protocol:openssl\ssl\s23_clnt.c:827:

It seems that we are going from a proxy error to some OpenSSL incompatibility. Do you have such OpenSSL error message before?

@valen22br
Copy link
Contributor

valen22br commented Aug 21, 2018

Hi @grant , the call to the method client.getToken(authCode) at auth.ts was not able to provide a return. Do you know how to debug that method?

@hanguokai
Copy link
Author

I don't know why env variables don't work like in the googleapis README - grant
Will a non-socks proxy work? - grant

I wrote a sample to test env variables for googleapis directly. I found googleapis itself has issue for supporting http(non-socks) proxy. Someone filed an issue here. Axios, googleapis use it as http client library, also has issue about http proxy.

So when googleapis fixed http proxy issue, let's review this issue again in clasp. Currently, I use VPN as a workaround method.

@valen22br
Copy link
Contributor

@hanguokai , good point in your research. My tests also revealed that apparently, the environment variables with proxy settings are working. The problem is when the clasp tries to use the token got from the authentication process to use it against the google API. In this step, it seems that someone, (Google API as you pointed) is messing around with https requests.
Here in our company, we have a single proxy that can handle both HTTP and https requests without distinction.

I'll take a look at the links you provided. Can you explain your workaround using the VPN behind the corporate proxy?

@google google deleted a comment from valen22br Aug 24, 2018
@google google deleted a comment from valen22br Aug 24, 2018
@google google deleted a comment from valen22br Aug 24, 2018
@grant
Copy link
Contributor

grant commented Aug 24, 2018

(I moved some off-topic comments to a separate thread to keep this about a proxy FR. See #297 (comment))

@aakasaka
Copy link

aakasaka commented Oct 3, 2018

It seems googleapis fixed the problem with proxy.
Could you please update the packages (googleapis and google-auth-library) to latest version?

@grant
Copy link
Contributor

grant commented Oct 4, 2018

I'm in the process of upgrading the packages.

@grant
Copy link
Contributor

grant commented Oct 4, 2018

Can you install the latest clasp, 1.6.1? LMK if it's fixed with the new googleapis module.
@hanguokai @valen22br @aakasaka @HeavensBlade

@aakasaka
Copy link

aakasaka commented Oct 5, 2018

I was able to clasp login and clasp create with the latest version, thank you !
FYI, I still should replace the code inOnline() with true coz is-online module doesn't work with proxy. (mentioned in #336 )

@grant
Copy link
Contributor

grant commented Oct 5, 2018

OK, thanks. I'll remove the isOnline feature with #332 since it's not super useful and blocks proxy users. Sorry @arjun-rao!

@valen22br
Copy link
Contributor

Tomorrow I will run some tests with a proxy configuration.
Thanks for the fix!

@grant
Copy link
Contributor

grant commented Oct 5, 2018

@aakasaka I removed the check to isOnline if HTTP_PROXY/HTTPS_PROXY is present with 1.6.3.
Let me know if a proxy works out of the box now, so I can close this issue.

@hanguokai
Copy link
Author

I simply tested clasp 1.6.3 with login, logout, status, clone, pull and push commands. It seems Ok now for http_proxy/https_proxy. Thanks for updating!

@grant
Copy link
Contributor

grant commented Oct 6, 2018

👍 Thanks for testing.
Closing this top clasp FR now!

@grant grant closed this as completed Oct 6, 2018
@zoudino
Copy link

zoudino commented May 6, 2021

Hi, @grant , could you provide a quick step by step tutorial on how to set up the proxy and use the clasp? I have been trying hours on figuring out this issue even after reading this,long discussion. The error I got is still showing "Error retrieving access token: FetchError: request to https://oauth2.googleapis.com/token failed, reason: connect ETIMEDOUT xxx:443"

@thedrs
Copy link

thedrs commented Jan 10, 2023

I solved this issue, as i had another problem here related to proxy - here is the story:

I am on windows in a very restricted corporate enterprise which has to use an http and https proxy.
We do have normal google services accessible in the browser.
Obviously 'clasp login' does not work without a proxy, it gives:
Error retrieving access token: FetchError: request to https://oauth2.googleapis.com/token failed, reason: connect ETIMEDOUT 216.58.212.202:443
(It does work if i disconnect my computer from corporate and connect to public internet)

So i defined these two env variables in cmd (you can also do it in git bash console, just replace "set" with "export"):
set HTTP_PROXY=http://proxy.mycompany.com:80
set HTTPS_PROXY=https://proxy.mycompany.com:80/
(our corporate proxy uses port 80 for both http and https !)

this is the error i got:
Error retrieving access token: FetchError: request to https://oauth2.googleapis.com/token failed, reason: write EPROTO EC110000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:c:\ws\deps\openssl\openssl\ssl\record\ssl3_record.c:355:

The solution to my problem was that our https proxy used an http address, so i fixed the set to the following :
set HTTPS_PROXY=http://proxy.mycompany.com:80](http://proxy.mycompany.com/)
(note the http:// and not https:// on the right hand side)

Hope i helped others here too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants