a04 Databases, logs, errors, and middleware

DUE: 2022-03-29T00:00:00-5:00

GitHub Classroom Assignment Invite: https://classroom.github.com/a/GQsrzvZT

Issues: https://github.com/comp426-2022-spring/a04/issues

Logging transactions and handling errors

This assignment for COMP 426 will help you to create and connect a database and to record access logs and test errors for your API server from a03 with some middleware. This is an important piece of building an API, because it allows you to break down what happened if something goes wrong. It also helps other developers use and deploy your software.

You should start with your server.js script from a03.

Documentation

The below code block should serve in part as a guide for this assignement.

If your server.js script is run with the option --help, it should echo ONLY the following help message to STDOUT and then exit 0.

Note that if you try to test this using npx nodemon server.js --help it will display nodemon’s help message. Also, in order for this to work properly, the conditional has to come before all of the dependencies in your script, with the exception of whatever library you are using to parse command line arguments (minimist, yargs, etc.).

server.js [options]

  --por		Set the port number for the server to listen on. Must be an integer
              	between 1 and 65535.

  --debug	If set to true, creates endlpoints /app/log/access/ which returns
              	a JSON access log from the database and /app/error which throws 
              	an error with the message "Error test successful." Defaults to 
		false.

  --log		If set to false, no log files are written. Defaults to true.
		Logs are always written to database.

  --help	Return this message and exit.

As you can see, you will need to include multiple command line options with defaults.

TIP: You can store help text in a variable spanning multiple lines by wrapping in with backticks.

If you are using minimist, to do this, look at the documentation to see how arguments are stored in the argv object that minimist gives you: https://github.com/substack/minimist

TIP: If you use minimist in the way that we have done in the past to process command line arguments (see below), you can see what it is storing in the object that we create from it by dumping into console log. This gives you access in real time for debugging.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// Require minimist module
const args = require('minimist')(process.argv.slice(2))
// See what is stored in the object produced by minimist
console.log(args)
// Store help text 
const help = (`
server.js [options]

--port	Set the port number for the server to listen on. Must be an integer
            between 1 and 65535.

--debug	If set to true, creates endlpoints /app/log/access/ which returns
            a JSON access log from the database and /app/error which throws 
            an error with the message "Error test successful." Defaults to 
            false.

--log		If set to false, no log files are written. Defaults to true.
            Logs are always written to database.

--help	Return this message and exit.
`)
// If --help or -h, echo help text to STDOUT and exit
if (args.help || args.h) {
    console.log(help)
    process.exit(0)
}
[email protected]:~/comp426/a04-johnmar3$ node server.js --debug --log=false --port=54321 --help
Command line arguments:  { _: [], debug: true, log: 'false', port: 54321, help: true }

server.js [options]

--port  Set the port number for the server to listen on. Must be an integer
            between 1 and 65535.
...

Creating a database

Use the methods that we have gone over in class to create a database file called log.db using better-sqlite3 with a table named accesslog. Set up the accesslog table with columns corresponding to variables listed below.

The notes from w08 should help with this.

Logging to database

Write a middleware function that inserts a new record in a database containing all of the variables in the following data object:

    let logdata = {
        remoteaddr: req.ip,
        remoteuser: req.user,
        time: Date.now(),
        method: req.method,
        url: req.url,
        protocol: req.protocol,
        httpversion: req.httpVersion,
        status: res.statusCode,
        referer: req.headers['referer'],
        useragent: req.headers['user-agent']
    }

NOTE: the above object previously contained a reference to secure: req.secure, but passing this to better-sqlite3 causes it to choke because the data cannot be coerced into a form that it will accept without a bunch of work. SO, skip it. It’s not important for the purposes of this assignment.

Remember that you have to include next() at the end of your middleware or it will hang.

And also remember to include it in the list of parameters passed to your middleware:

app.use( (req, res, next) => {
// Your middleware goes here.
})

Create endpoints

These endpoints should be available if AND ONLY IF the command line option --debug=true is passed when server.js is run.

/app/log/access

Create a new endpoint /app/log/access that returns all records in the accesslog table in your database log.db.

/app/error

Create an endpoint /app/error that returns an error in the response.

The error should read “Error test successful.”

HINT: This should just work if you model your endpoint on the first example in the error documentation from Express: http://expressjs.com/en/guide/error-handling.html

Create an access log file

In addition to the log in your database, if --log=false is passed when running server.js, then your server should NOT create a log file. Otherwise, your server should create a FILE called access.log and write your access log to it in a combined FORMAT (default behavior).

Use morgan to accomplish this by sending its output to a write stream from the fs builtin:

1
2
3
4
5
6
7
...
// Use morgan for logging to files
// Create a write stream to append (flags: 'a') to a file
const WRITESTREAM = fs.createWriteStream('FILE', { flags: 'a' })
// Set up the access logging middleware
app.use(morgan('FORMAT', { stream: WRITESTREAM }))
...

You will need to change everything in ALLCAPS above to suit the instructions.

List of requirements

  1. Return help message to STDOUT and exit 0 if command line argument --help is passed.
  2. Use better-sqlite3 to create a database log.db and populate it with one table accesslog with the variables listed above in the logdata object.
  3. Write a middleware function that writes the logdata object variables into the accesslog table in log.db.
  4. Create endpoints /app/log/access and /app/error that send a response in the form of an access log and an error test, respectively, when --debug is passed as a command line argument.
  5. Write a combined format access log to an access.log file when --log is passed as a command line argument.
  6. As usual, include a command line argument to set an arbitrary port number that defaults to 5555.