Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support \t in Content-Type header #808

Open
rg2011 opened this issue Jan 30, 2024 · 2 comments
Open

Support \t in Content-Type header #808

rg2011 opened this issue Jan 30, 2024 · 2 comments
Labels
bug NGSIv2

Comments

@rg2011
Copy link
Contributor

rg2011 commented Jan 30, 2024 •
edited
Loading

IoT Agent JSON version the issue has been seen with

3.1.0

Bound or port used (API interaction)

Southbound (Devices data API)

NGSI version

NGSIv2

Are you running a container?

Yes, I am using a contaner (Docker, Kubernetes...)

Image type

normal

Expected behaviour you didn't see

This is a follow-up to issue #759. The XML service integration still fails because the client includes a "\t" character in the Content-Type header.

When sending a measure with content-type "application/soap xml;\tcharset=utf-8" to /iot/json/attrs/<name of attrib>, the request should be accepted according to #771

Unexpected behaviour you saw

The IoT Agent replies inmediately with "400 Bad Content".

This is because the type-is module, used internally by express, fails to match this content-type to application/soap xml:

$ node
Welcome to Node.js v20.10.0.
Type ".help" for more information.
> const typeis = require('type-is').is
undefined
> const ctype = "application/soap xml; action=\"urn:notificarEvento\"; \tcharset=utf-8"
undefined
> typeis(ctype, ["application/soap xml"])
false
> typeis(ctype, ctype)
false
>

This causes the following checks to fail:

iotagent-json/lib/bindings/HTTPBinding.js

Lines 75 to 83 in db40299

if (req.is('text/plain')) {
text(req, res).then(() => next(), next);
} else if (req.is('application/octet-stream')) {
raw(req, res).then(() => next(), next);
} else if (req.is('application/soap xml')) {
xml(req, res).then(() => next(), next);
} else {
// req.is('json')
json(req, res).then(() => next(), next);

iotagent-json/lib/bindings/HTTPBinding.js

Lines 109 to 115 in db40299

if (
req.method === 'POST' &&
!req.is('json') &&
!req.is('text/plain') &&
!req.is('application/octet-stream') &&
!req.is('application/soap xml')
) {

And also an internal test in the bodyparser-xml dependency, which performs its own content-type check, matching to ['*/xml', ' xml'] by default:

Removing the \t makes type-is recognize the content-type properly:

> const ctype2 = "application/soap xml; action=\"urn:notificarEvento\"; charset=utf-8"
undefined
> typeis(ctype2, ["application/soap xml"])
'application/soap xml'
>

I did open an issue upstream: jshttp/type-is#52

But I don't expect a fix to reach express anytime soon, so maybe we would need a workaround.

Steps to reproduce the problem

export TAB=$'\t'

curl -H "Content-Type: application/soap xml;${TAB}charset=utf-8" "http://<iota json ip>:7897/iot/json/attrs/payload?i=<deviceid>&k=<apikey>" -d '<?xml version="1.0"?><body><attrib>value</attrib></body>"

Configs

environment:
    - IOTA_APPEND_MODE=true
    - IOTA_MONGO_HOST=iot-mongo
    - IOTA_MONGO_DB=iotajson
    - IOTA_SINGLE_MODE=false
    - IOTA_CB_HOST=iot-orion
    - IOTA_CB_NGSI_VERSION=v2
    - IOTA_AUTH_ENABLED=false
    - IOTA_AUTH_TYPE=keystone
    - IOTA_AUTH_HEADER=X-Auth-Token
    - IOTA_AUTH_HOST=172.17.0.1
    - IOTA_AUTH_PORT=5001
    - IOTA_AUTH_USER=iotagent
    - IOTA_AUTH_PASSWORD=***OFUSCATED***
    - IOTA_IOTAM_HOST=iot-iotagent-manager
    - IOTA_IOTAM_PORT=8082
    - IOTA_IOTAM_PATH=/iot/protocols
    - IOTA_IOTAM_AGENTPATH=
    - IOTA_IOTAM_PROTOCOL=IoTA-JSON
    - IOTA_IOTAM_DESCRIPTION=JSON_IoT_Agent_Node
    - IOTA_PROVIDER_URL=http://172.17.0.1:4052
    - IOTA_NORTH_PORT=4052
    - IOTA_HTTP_PORT=7897
    - IOTA_MQTT_USERNAME=iota
    - IOTA_MQTT_PASSWORD=***OFUSCATED***
    - IOTA_MQTT_HOST=iot-mosquitto
    - IOTA_MQTT_PORT=1883
    - IOTA_MQTT_QOS=2
    - IOTA_AMQP_DISABLED=true
    - IOTA_LOG_LEVEL=INFO
    - IOTA_REGISTRY_TYPE=mongodb
    - IOTA_DEFAULT_RESOURCE=/iot/json

Log output

{"name":"UNSUPPORTED_TYPE","message":"The request content didn't have the expected type [application/json, text/plain, application/octet-stream, application/soap xml ]"}
@rg2011 rg2011 added the bug label Jan 30, 2024
@AlvaroVega
Copy link
Member

I've tested with

req.get('content-type').includes('application/soap xml')

but doesn't work.

@rg2011
Copy link
Contributor Author

rg2011 commented Feb 5, 2024

I've tested with

req.get('content-type').includes('application/soap xml')

but doesn't work.

Hi Alvaro. I understand you were able to replicate the issue with a test, or something like the snippet:

export TAB=$'\t'

curl -H "Content-Type: application/soap xml;${TAB}charset=utf-8" "http://<iota json ip>:7897/iot/json/attrs/payload?i=<deviceid>&k=<apikey>" -d '<?xml version="1.0"?><body><attrib>value</attrib></body>"

But changing req.is to req.get('content-type').includes(...) was not enough to fix it. Is that right?

I am working in a forked branch, and found some other changes are needed too. Let me open a PR so we can discuss the details more easily.

@rg2011 rg2011 mentioned this issue Feb 5, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug NGSIv2
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@AlvaroVega @rg2011