In Fall 2024 my team in CSC309 scored one of the highest marks in the course project (104%). The technical challenge was to build a web app which allowed for executing code, displaying blogs, adding reviews, storing templates, and adding content. Each student was in one of approximately 75 teams of 3.

We needed to implement a database for storing users and then encrypt/decrypt user-entered passwords so that they would be able to access the site every time. In addition, we needed to make the site mobile responsive for all pages. There were two milestones (part 1 and part 2) where our team scored a grand total of 104% (after bonus marks).

Scriptorium is an innovative online platform where you can write, execute, and share code in multiple programming languages. Inspired by the ancient concept of a scriptorium, a place where manuscripts were crafted and preserved, Scriptorium modernizes this idea for the digital age. It offers a secure environment for geeks, nerds, and coding enthusiasts to experiment, refine, and save their work as reusable templates. Whether you’re testing a quick snippet or building a reusable code example, Scriptorium is what you need to bring your ideas to life.

There are three types of users in the application. The first type is Visitor which uses no authentication. User and Admin roles require authentication and user accounts to be created for each.

Visitors to the blog site have no authentication and are able to access requests including the endpoints in /api/execute, fork a template, and browse/read blogs.

User level users can save templates, create, delete, and edit blog posts, and provide comments/rate content. Admin level permissions allow users to sort by reports and review reported inappropriate content.

The application is deployed to a Digital Ocean droplet. It utilizes Docker to conduct virtualization and isolation. On the server, NGINX is used as a server manager. We use PM2 for process management and the site is deployed on the server to localhost:3000 and NGINX acts as a reverse proxy. We also enabled firewall functionalities through ufw to ensure only intended applications can access various backend endpoints.

The site has a Let’s Encrypt SSL which automatically updates so that requests can be served through HTTPS. The site is deployed to https://codeshare.ca/.

Homepage of our application

The tech stack consists of Typescript, NextJS, Docker (YAML), JS, and a Prisma DB running SQLite. The cool thing about this project is that it is a monolith design meaning that both the frontend and the backend are deployed from the same root folder and hosted on the same server. This is thanks to Next JS which allows us to build a functional REST API backend easily all in JS.

Although there are some security concerns about this methodology, we have ensured completely isolated containerization of executions. In addition, we enabled docker containers to be killed immediately after 10 seconds and after memory overflows to ensure container isolation.

Executions in C

Technical Details #

Below I explain some of our project’s technical implementations, challenges, and solutions we came up for the project.

How we get the Executions to Work #

For executing different languages of code the obvious problem is that the browser cannot handle different languages without interacting with a server. The browser is running a node instance and thus we can’t run python or c in the browser. The code for c also needs to be compiled first before being executed while python is simply interpreted. Therefore, we need to turn to other technologies. A naive approach for this would be to allow the user to execute the code on the bare-bone operating system. However, this poses many challenges, namely security related challenges. We cannot afford to allow the user to write malicious scripts without isolating their scripts completely from our operating system. In addition, it’s difficult to ensure that the code being run won’t block other processes on the operating system or even hog resources. Thanks to my experience with Docker in my internships, I knew what to do to get the system to work. We needed to have Docker images ready to be run and then create a JS thread to generate a container for each time someone wanted to execute code. The container would execute and return the STD Errors and STD Output back to the JS through Async functions. We ended up porting a setup.js which allowed us to download all the required docker images beforehand into the docker environment. When a user actually wanted to execute code we’d compile if necessary, then start up a container for their isolated execution.

Another approach would have been to instantiate separate virtual machines for every instance. The main drawback of this approach is that the executions now require substantial overhead to begin and teardown. The scripts that users are running are generally very small scripts and so there is not a need for us to run an entire virtual machine with its own operating system.

How we managed login/logout #

In lectures, we learned about using jsonwebtoken and bcrypt for authentication and session management. We used these ideas for our website to have a fully functional login/logout mechanism for all users.

For password management, we hashed the passwords for users to login so that even system admins do not know what the passwords users have entered are. Then, when verifying the password, we hash their password again and then compare their entered password to the hashed version. We also maintain a session token which expires every 7 days and a refresh token which expires every hour. This allows us to know who is using the site and which people we need to allow to log in/out and at what time.

The login screen for users to login

After logging in this is what the creator studio looks like

How we handle standard input into the executions #

We handle standard input into the execution pane by also passing build flags into the docker run command. This allows us to enable users to pass in standard input into the code they want to execute.

For instance, for c we have runCommand = gcc -Wall main.c -o main && ./${cExecutablePath};. This first compiles main.c then converts it into an executable.

Python stdin is an example of the stdin functionalities

Which languages we supported #

We used the Docker images for each of the languages so that we didn’t need to download the languages every time we ran. Rather, we have a setup script that downloads pre-selected languages and their versions. The supported languages were Java, Javascript, Python, C, C++, Lua, Rust, Ruby, PHP, and Perl. The requirement was to pick at least 10 languages and so we picked languages which we were familiar wth and readily available.

Our list of supported code and their versions are listed below

Our run.sh also downloads the correct versions before executions begin.

        python: "python:3.10",
        javascript: "node:20",
        java: "openjdk:20",
        cpp: "gcc:12",
        c: "gcc:12",
        ruby: "ruby:3.3",
        php: "php:8.2-cli",
        perl: "perl:5.36",
        lua: "nickblah/lua",
        rust:"rust:1.73",

For instance, docker python 3.10 can be found at https://hub.docker.com/_/python.

Handling Timeouts and Memory #

Our server also handled timeouts and errors with memory. For instance, if we ran a while loop that never ended, our Docker service would be able to detect that and then kill the container. Our backend would tell the frontend that there are errors with the last execution and display an error message.

We implemented this by checking for specific Docker error codes returned. For the memory error, we looked for ERR_CHILD_PROCESS_STDIO_MAXBUFFER and for the timeout, we looked for error.code == 124. Once we noticed these errors, we returned a proper error message to the frontend to display to the user.

We handled timeouts and also errors with memory.

How to link templates with blogs #

I also had a lot of experience with database design during my internships and was able to help our team design a functional database backend. This backend contained methods for people to query for various database objects like blogs, templates, and more. I put all the tests into Postman so that we could show our TAs and each other which ones were working.

A major challenge was in getting the tags to work, especially tags with multiple spaces. This is because searching required us to search for text rather than just compare tags.

Blogs are able to be added and edited

Templates #

I was also in charge of the templates section. I played around with getting the code viewer working to enable correct syntax highlighting for the code. The templates were a bit complicated in that I needed to have the ability for users to actually execute the code of the template. I ended up making a separate path for just executing the code for the template. So https://www.codeshare.ca/code/1 corresponds to https://www.codeshare.ca/templates/1.

We were very grateful that we could use Prisma because of the ORM provided out-of-the-box ability for us to easily use functions such as search. I was able to connect the search endpoints into the backend so that the users could search for blogs very quickly.

I also handled the forking of templates so that users could use it on their own.

Templates allow users to save their code for others to use

Deployment #

I was in charge of deployment. This got us the bonus marks. The basic marks were for completing all the tasks in the rubric, but these bonus marks I thought would be super easy to implement. I first got a domain from Namecheap (around $10). Then, I pointed the nameservers to my Digital Ocean server running all my course projects. I configured NGINX to allow requests to the codeshare.ca domain to be let through. And I ran my Docker setup on my server to start the service. I also configured PM2 to run in the background and after every restart. Finally, I got Let’s Encrypt to do automatic SSL encryption. All in all, the entire project costs less than $20 a year to operate.

Things my teammates did which I think were pretty cool #

• We have an entire blog/comments section which allows almost infinite recursion in commenting on blogs. Users can keep on adding comments to blogs.
• Our project incorporates admin/user functionalities. Admins can hide content which they deem suspicious and also delete some content.
• We have the ability for users to switch between different avatars.
• Responsive for both mobile, tablet, and desktop.
• Ability to switch between light and dark modes.

Here you can see that we handled extensive comment recursion and also our reporting feature.

Things we would have done with more time #

Given more time, we would have liked to implement the following:

• Images in the blogs so that users could see images and post about them there.
• Additional resources for users to learn the languages/suggest languages.
• Change avatars to uploaded photos.

Executions in Java

What I Learned #

Overall, this project was really great as I learned the foundations of web development and deploying applications. Some things I learned or were able to solidify my understanding of.

• Nginx
• React
• Backend/Frontend
• Monolithic Architectures
• REST API development
• Deployment
• Docker
• Docker Compose
• PM2
• Authorization and Authentication

I thought the course was excellently taught by Kianoosh and I learned a ton. I think combining it with other systems courses makes for an excellent learning experience.

The GitHub repository of our site is available at https://github.com/vzcodes/Scriptorium. Teammates were Will and Abdullah.