Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Telegram integration error "Flood control exceeded" #1103

Closed
aksenk opened this issue Jan 7, 2023 · 7 comments · Fixed by #2100
Closed

Telegram integration error "Flood control exceeded" #1103

aksenk opened this issue Jan 7, 2023 · 7 comments · Fixed by #2100

Comments

@aksenk
Copy link

aksenk commented Jan 7, 2023

Hi!

I have a problem with configuring telegram integration.

I have deployed grafana oncall via helm chart and installed telegram token and telegram webhookUrl (like https://oncall.mydomain.com/).

But when I open "ChatOps -> Telegram" page I see message: To connect channel setup Telegram environment first, which includes connection to your bot and host URL.

If I go to the page "Env Variables" I see that my variables are filled in, but in column Status for variable TELEGRAM_WEBHOOK_HOST I see error: Telegram error: Flood control exceeded. Retry in 1.0 seconds.

I don't understand what's wrong, because I created a new bot for a new installation (which doesn't even have a single alert yet).

Helm chart version: 1.1.0
Oncall app version: 1.1.5
Grafana oncall plugin version: 1.1.14
Grafana version: 9.2.4

Please help.

UPD1: I tried to install a compatible versions of oncall app and oncall plugin (both versions 1.1.5) and it didn't help.

UPD2: I rolled back both versions to 1.1.0 and have the same problem

UPD3: It works correctly when I rolled back to version 1.0.51
it looks like this functionality has been broken since v1.1.0

@Sammyant
Copy link

Sammyant commented Jan 26, 2023

Hi, we have the same issue with the new version (v1.1.19) in docker compose

@joanyko
Copy link

joanyko commented Jan 29, 2023

Hi, same issue in helm chart (v1.1.0)

@Acetolyne
Copy link

Acetolyne commented Jan 31, 2023

please try removing the trailing / from the url I noticed when setting mine up that i could not have the trailing / in the url else I got the same message. If this fixes the issue for you please let us know and I can put in a PR to automatically remove the trailing slash so that the plugin handles these URLs in a better way.

Note that this can also be caused by the url not being publicly accessible by telegram please confirm that the url you are putting in is accessible in your web browser and returns Ok on the page that is served

@kirpozh
Copy link

kirpozh commented Feb 21, 2023

The problem still persists. I tried to proxy traffic through a separate endpoint, I also tried through ngrok, as described in the tutorial video - it does not work.
Removing trailing from the message above didn't help either.
As described in the first message, that only downgrade to a version lower than 1.0.51 works, then I can't do it because of the inability to put this version through compose.
Please help

@vitvord
Copy link

vitvord commented Feb 24, 2023

We have the same problem. Removing the database and completely reinstalling redis/rabbitmq/oncall does not help.
webhookUrl is publicly available.

oncall app and plugin: v1.1.29

@kuzaxak
Copy link

kuzaxak commented May 14, 2023

Version: 1.2.22

Same behaviour without tailing /

@alexintech
Copy link
Contributor

I had the same problem. I suppose if for some period TELEGRAM_WEBHOOK_HOST was wrong or publicly unavailable, such error may occur and oncall saves it into database.

When I change TELEGRAM_WEBHOOK_HOST in UI, it successfully registers webhook. It can be verified by using:

curl --location --request POST 'https://api.telegram.org/bot{{token}}/getWebhookInfo'

You should see "url": "{{TELEGRAM_WEBHOOK_HOST}}/telegram/", and if you type some message to your bot, in oncall-engine's logs you will see that message is received by oncall.

The problem is when you change TELEGRAM_WEBHOOK_HOST in UI, it does not clear previous error in database. And other telegram UI functionality looks at the TELEGRAM* variables without errors:

https://github.com/grafana/oncall/blob/f18858882eb8239ec6cfa844c7d676b4125a1211/engine/apps/api/serializers/organization.py#LL117C19-L117C19

I have cleaned error value in base_livesetting table, and after that all is working

update base_livesetting set error = null where name = 'TELEGRAM_WEBHOOK_HOST';

Hope it will help to fix an issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants