Awesome
parse-request
Parse requests in the Browser and Node (with added support for multer and passport). Made for Cabin.
Table of Contents
Install
npm:
npm install parse-request
How does it work
This package is used internally by Cabin's middleware, and we highly recommend you to simply use Cabin instead of this package in particular.
This package exports a function that accepts an Object options
argument:
options
(Object) - a configuration objectreq
(Object) - an Express/Connect request object (defaults tofalse
) (you must either pass this option orctx
option, but not both)ctx
(Object) - a Koa context object (defaults tofalse
) (you must either pass this option orreq
option, but not both)responseHeaders
(String or Object) - we highly recommend that you passres._headers
(see Cabin's middleware logic if you need an example of this)userFields
(Array) - defaults to[ 'id', 'email', 'full_name', 'ip_address' ]
, list of fields to cherry-pick from the user object parsed out ofreq.user
sanitizeFields
(Array) - defaults to the list of Strings provided under Sensitive Field Names Automatically Masked belowsanitizeHeaders
(Array) - defaults to the list of Strings provided under Sensitive Header Names Automatically Masked below (case insensitive)maskCreditCards
(Boolean) - defaults totrue
, and specifies whether or not credit card numbers are maskedmaskBuffers
(Boolean) - defaults totrue
, and will rewriteBuffer
's,ArrayBuffer
's, andSharedArrayBuffer
's recursively as an object of{ type: <String>, byteLength: <Number> }
. Note that this will save you on disk log storage size as logs will not output verbose stringified buffers – e.g. imagine a 10MB file image upload sent across the request body as a Buffer!)maskStreams
(Boolean) - defauls totrue
, and will rewriteStream
's to{ type: 'Stream' }
(this is useful for those using multer v2.x (streams version), or those that have streams inreq.body
,req.file
, orreq.files
)checkId
(Boolean) - defaults totrue
, and prevents Strings that closely resemble primary key ID's from being masked (e.g. properties named_id
,id
,ID
,product_id
,product-id
,productId
,productID
, andproduct[id]
won't get masked or show as a false-positive for a credit card check)checkCuid
(Boolean) - defaults totrue
, and prevents cuid values from being maskedcheckObjectId
(Boolean) - defaults totrue
, and prevents MongoDB BSON ObjectId from being maskedcheckUUID
(Boolean) - defaults totrue
, and prevents uuid values from being maskedrfdc
(Object) - defaults to{ proto: false, circles: false }
(you should not need to customize this, but if necessary refer to rfdc documentation)parseBody
(Boolean) - defaults totrue
, if you set tofalse
we will not parse nor clone the requestbody
property (this overrides all other parsing settings related)parseQuery
(Boolean) - defaults totrue
, if you set tofalse
we will not parse nor clone the requestquery
property (this overrides all other parsing settings related)parseFiles
(Boolean) - defaults totrue
, if you set tofalse
we will not parse nor clone the requestfile
norfiles
properties (this overrides all other parsing settings related)
It automatically detects whether the request is from the Browser, Koa, or Express, and returns a parsed object with populated properties.
Here's an example object parsed:
{
"id": "5d126d86160cea56950f80a9",
"timestamp": "2019-06-25T18:52:54.000Z",
"is_http": true,
"request": {
"method": "POST",
"query": {
"foo": "bar",
"beep": "boop"
},
"headers": {
"host": "127.0.0.1:59746",
"accept-encoding": "gzip, deflate",
"user-agent": "node-superagent/3.8.3",
"authorization": "Basic ********************",
"accept": "application/json",
"cookie": "foo=bar;beep=boop",
"content-type": "multipart/form-data; boundary=--------------------------104476455118209968089794",
"content-length": "1599",
"connection": "close"
},
"cookies": {
"foo": "bar",
"beep": "boop"
},
"url": "/?foo=bar&beep=boop",
"body": "{\"product_id\":\"5d0350ef2ca74d11ee6e4f00\",\"name\":\"nifty\",\"surname\":\"lettuce\",\"bank_account_number\":\"1234567890\",\"card\":{\"number\":\"****-****-****-****\"},\"stripe_token\":\"***************\",\"favorite_color\":\"green\"}",
"timestamp": "2019-06-25T18:52:54.589Z",
"id": "fbbce5d4-02d9-4a81-9a70-909631317e7d",
"http_version": "1.1",
"files": "{\"avatar\":[{\"fieldname\":\"avatar\",\"originalname\":\"avatar.png\",\"encoding\":\"7bit\",\"mimetype\":\"image/png\",\"buffer\":{\"type\":\"Buffer\",\"byteLength\":216},\"size\":216}],\"boop\":[{\"fieldname\":\"boop\",\"originalname\":\"boop-1.txt\",\"encoding\":\"7bit\",\"mimetype\":\"text/plain\",\"buffer\":{\"type\":\"Buffer\",\"byteLength\":7},\"size\":7},{\"fieldname\":\"boop\",\"originalname\":\"boop-2.txt\",\"encoding\":\"7bit\",\"mimetype\":\"text/plain\",\"buffer\":{\"type\":\"Buffer\",\"byteLength\":7},\"size\":7}]}"
},
"user": {
"ip_address": "::ffff:127.0.0.1"
},
"response": {
"headers": {
"x-powered-by": "Express",
"x-request-id": "fbbce5d4-02d9-4a81-9a70-909631317e7d",
"content-security-policy": "default-src 'none'",
"x-content-type-options": "nosniff",
"content-type": "text/html; charset=utf-8",
"content-length": "1213",
"x-response-time": "48.658ms",
"date": "Tue, 25 Jun 2019 18:52:54 GMT",
"connection": "close"
},
"http_version": "1.1",
"status_code": 200,
"reason_phrase": "OK",
"timestamp": "2019-06-25T18:52:54.000Z",
"duration": 48.658
},
"duration": 1.350323,
"app": {
"name": "parse-request",
"version": "1.0.11",
"node": "v10.15.3",
"hash": "f99bb8f28be5c6dc76bed76f6dd8984accc5c5fa",
"environment": "test",
"hostname": "users-MacBook-Pro.local",
"pid": 22165
}
}
A few extra details about the above parsed properties:
id
(String) - is a newly created BSON ObjectId used to uniquely identify this logtimestamp
(String) - is the ISO-8601 date time string parsed from theid
(thanks to MongoDB BSONObjectID.getTimestamp
method)is_http
(Boolean) - defaults totrue
to indicate that this was a parsed HTTP requestduration
(Number) - is the number of milliseconds thatparseRequest
took to parse the request object (note that this usesprocess.hrtime
which this package polyfills thanks to browser-process-hrtime)user
(Object) - is parsed from the user object onreq.user
automatically (e.g. you are using passport):ip_address
(String) - IP address parsed...
- additional fields are optionally parsed fromreq.user
request
(Object) - request object information parsed fromoptions.req
:id
(String) - is conditionally added ifreq.id
is a String (we highly recommend that you use express-request-id in your project, which will automatically add this property ifX-Request-Id
if it is set, otherwise it will generate it as a new UUID)file
(Object) - is conditionally added if you have areq.file
property (e.g. if you're using multer)files
(Array) - is conditionally added if you have areq.files
property (e.g. if you're using multer)http_version
(String) - is parsed fromreq.httpVersion
orreq.httpVersionMajor
andreq.httpVersionMinor
timesamp
(String) - is the ISO-8601 date time string parsed from when the request was received (we highly recommend that you use request-received for this to be parsed as accurately as possible, although we do support a few widely-used fallback approaches)headers
(Object) - the raw request headers (lowercased)
response
(Object) - response object information parsed fromoptions.responseHeaders
(we use http-headers to parse this information):http_version
(String) - is parsed from the response HTTP headers major and minor HTTP versiontimestamp
(String) - is the ISO-8601 date time string parsed from the response'sDate
headerduration
(Number) - is the number of milliseconds parsed from theX-Response-Time
HTTP header (we highly recommend that you use response-time and request-received)headers
(Object) - the raw response headers (lowercased)status_code
(Number) - the response's status code (see RFC spec on Status Code and Reason Phrase)reason_phrase
(String) - the response's reason phrase (see RFC spec on Status Code and Reason Phrase)
Please see Credit Card Masking and Sensitive Field Names Automatically Masked below for more information about how request.body
, request.file
, and request.files
are parsed and conditionally masked for security.
Credit Card Masking
We also have built-in credit-card number detection and masking using the credit-card-type library.
This means that credit card numbers (or fields that are very similar to a credit card) will be automatically masked. If you'd like to turn this off, pass false
to maskCreditCards
**
Sensitive Field Names Automatically Masked
See sensitive-fields for the complete list.
Sensitive Header Names Automatically Masked
The Authorization
HTTP header has its <credentials>
portion automatically masked.
This means that if you are using BasicAuth or JSON Web Tokens ("JWT"), then your tokens will be hidden.
Usage
We highly recommend to simply use Cabin as this package is built-in!
VanillaJS
The example below uses xhook which is used to intercept HTTP requests made in the browser.
<script src="https://cdnjs.cloudflare.com/polyfill/v3/polyfill.min.js?features=performance,WeakRef"></script>
<script src="https://unpkg.com/xhook"></script>
<script src="https://unpkg.com/parse-request"></script>
<script type="text/javascript">
(function() {
xhook.after(function(req, res) {
var req = parseRequest({ req });
console.log('req', req);
// ...
});
})();
</script>
Required Browser Features
We recommend using https://cdnjs.cloudflare.com/polyfill (specifically with the bundle mentioned in VanillaJS above):
<script src="https://cdnjs.cloudflare.com/polyfill/v3/polyfill.min.js?features=performance,WeakRef"></script>
- performance.now() is not supported in op_mini all
- WeakRef is not supported in Opera 91
Koa
const parseRequest = require('parse-request');
// ...
app.get('/', (ctx, next) => {
const req = parseRequest({ req: ctx });
console.log('req', req);
// ...
});
Express
const parseRequest = require('parse-request');
// ...
app.get('/', (req, res, next) => {
const req = parseRequest({ req });
console.log('req', req);
// ...
});
If you override req.body and need to preserve original in logs
Sometimes developers overwrite req.body
or req.body
properties – therefore if you want to preserve the original request, you can add req._originalBody = req.body
(or ctx.request._originalBody = ctx.request.body
if you're using Koa) at the top of your route middleware (or as a global route middleware).
If you want to disable body parsing just for a specific route (e.g. prevent log output from showing the body)
If you're using Express:
const disableBodyParsing = Symbol.for('parse-request.disableBodyParsing');
// ...
app.get('/', (req, res, next) => {
req[disableBodyParsing] = true;
next();
});
If you're using Koa:
const disableBodyParsing = Symbol.for('parse-request.disableBodyParsing');
// ...
app.get('/', (ctx, next) => {
ctx.req[disableBodyParsing] = true;
next();
});
If you want to disable file parsing just for a specific route (e.g. prevent log output from showing the file(s))
If you're using Express:
const disableFileParsing = Symbol.for('parse-request.disableFileParsing');
// ...
app.get('/', (req, res, next) => {
req[disableFileParsing] = true;
next();
});
If you're using Koa:
const disableFileParsing = Symbol.for('parse-request.disableFileParsing');
// ...
app.get('/', (ctx, next) => {
ctx.req[disableFileParsing] = true;
next();
});
Contributors
Name | Website |
---|---|
Nick Baugh | http://niftylettuce.com/ |