# js-framework-benchmark **Repository Path**: mirrors_krausest/js-framework-benchmark ## Basic Information - **Project Name**: js-framework-benchmark - **Description**: A comparison of the performance of a few popular javascript frameworks - **Primary Language**: Unknown - **License**: Apache-2.0 - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 1 - **Created**: 2022-01-05 - **Last Updated**: 2025-08-17 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # js-framework-benchmark This is a simple benchmark for several javascript frameworks. The benchmarks creates a large table with randomized entries and measures the time for various operations including rendering duration.  ## Security advice Currently there are 186 implementations in this repository. It's of course impossible for me to make a security assessment for all those implementations. `npm ci` and `npm install` can execute arbitraty commands, so they should be executed only for packages you trust. Consequently I build on a dedicated virtual private linux server such that I don't have to install the packages for all those implemenations on my laptop. There's a prebuild build.zip for each chrome release you can download such that you can avoid installing the packages from all implementations. (I don't know of any (attempted) case for malicious packages in this repository, so please take it just as a general warning.) The server implemenation in this repository should only be started on your local machine and access should be restricted to your local machine. I recommend against starting the server such that it can be publically accessed from the internet. ## About the benchmarks The following operations are benchmarked for each framework: - create rows: Duration for creating 1,000 rows after the page loaded (no warmup). - replace all rows: Duration for replacing all 1,000 rows of the table (with 5 warmup iterations). - partial update: Time to update the text of every 10th row for a table with 10,000 rows (with 5 warmup iterations). - select row: Duration to highlight a row in response to a click on the row. (with 5 warmup iterations). - swap rows: Time to swap 2 rows on a table with 1,000 rows. (with 5 warmup iterations). - remove row: Duration to remove a row for a table with 1,000 rows. (with 5 warmup iterations). - create many rows: Duration to create 10,000 rows (no warmup) - append rows to large table: Duration for adding 1,000 rows on a table of 10,000 rows (no warmup). - clear rows: Duration to clear the table filled with 10,000 rows. (no warmup) - ready memory: Memory usage after page load. - run memory: Memory usage after adding 1,000 rows. - update memory: Memory usage after clicking 5 times update for a table with 1,000 rows. - replace memory: Memory usage after clicking 5 times create 1,000 rows. - repeated clear memory: Memory usage after creating and clearing 1,000 rows for 5 times. - update memory: Memory usage after clicking 5 times update for a table with 1,000 rows. - startup time: Duration for loading and parsing the javascript code and rendering the page. - consistently interactive: The lighthouse metric TimeToConsistentlyInteractive: A pessimistic TTI - when the CPU and network are both definitely very idle. (no more CPU tasks over 50ms) - script bootup time: The lighthouse metric ScriptBootUpTtime: The total ms required to parse/compile/evaluate all the page's scripts - main thread work cost: The lighthouse metric MainThreadWorkCost: Total amount of time spent doing work on the main thread includes style/layout/etc. - total byte weight: The lighthouse metric TotalByteWeight: Network transfer cost (post-compression) of all the resources loaded into the page. For all benchmarks the duration is measured including rendering time. You can read some details on this [article](http://www.stefankrause.net/wp/?p=218) and in the [wiki](https://github.com/krausest/js-framework-benchmark/wiki/How-the-duration-is-measured). Starting with chrome 118 the overall performance is computed as a [weighted geometric mean](https://github.com/krausest/js-framework-benchmark/wiki/Computation-of-the-weighted-geometric-mean). ## Official results Official results are posted on the [official results page](https://krausest.github.io/js-framework-benchmark/index.html). My [blog](http://www.stefankrause.net/wp) has a few articles about the benchmark. Older results of this benchmark are outlined on my blog ([round 1](http://www.stefankrause.net/wp/?p=191), [round 2](http://www.stefankrause.net/wp/?p=283), [round 3](http://www.stefankrause.net/wp/?p=301), [round 4](http://www.stefankrause.net/wp/?p=316), [round 5](http://www.stefankrause.net/wp/?p=392), [round 6](http://www.stefankrause.net/wp/?p=431), [round 7](http://www.stefankrause.net/wp/?p=454) and [round 8](http://www.stefankrause.net/wp/?p=504)). ## Snapshot of the results The current snapshot that may not have the same quality (i.e. results might be for mixed browser versions, number of runs per benchmark may vary) can be seen [here](https://krausest.github.io/js-framework-benchmark/current.html) [](https://krausest.github.io/js-framework-benchmark/current.html) ## Keyed vs non-keyed frameworks Some frameworks like React, Vue.js or Angular, allow you to create a 1:1 relationship between a data item and a DOM node by assigning a “key” attribute (or for Angular, specifying “trackBy” in *ngFor). If you use some identifier of the data as the key, you get the “keyed” mode. Any update to the data will update the associated DOM node. If you reorder the list, the DOM nodes will be reordered accordingly. The other mode is “non-keyed” and this is what e.g. vue.js uses by default for lists. In this mode, a change to the data items can modify DOM nodes that were associated with other data before. This can be more performant, since costly DOM operations can be avoided (e.g. first removing old nodes and then adding new nodes) and the existing DOM nodes are updated to display the new data. For React and Angular, using the item index as the key uses “non-keyed” mode for those frameworks. Depending on your requirements, the “non-keyed” mode can be a performance gain or can cause severe problems, so one must carefully choose the mode and check that the framework supports that mode. Read more here: [https://www.stefankrause.net/wp/?p=342](https://www.stefankrause.net/wp/?p=342) # 1 Run pre-built binaries for all frameworks There are currently 186 implementations in this repository. Installing (and maintaining) those can be challenging, but here are simplified instructions how to get started. See the security advice above to read why that might be a good idea. ## 1.1 Prerequisites Have _node.js (>=v20.9.0)_ installed. If you want to do yourself a favour use nvm for that. The benchmark has been tested with node v20.9.0. Please make sure that the following command work before trying to build: ``` > npm npm -version 10.1.0 > node --version v20.9.0 ``` ## 1.2 Downloading the pre-built binaries and starting the server building all frameworks can be challenging. There's a new way that allows to skip that and just run the benchmark without building all implementations. Start with checking out a tagged release like that. Pick the release that you want (e.g. chrome 100): ``` git clone https://github.com/krausest/js-framework-benchmark.git cd js-framework-benchmark git checkout chrome100 -b release npm ci && npm run install-local ``` Download the build.zip for that release from https://github.com/krausest/js-framework-benchmark/releases and put the build.zip into the js-framework-benchmark directory and unzip the prebuilt files: ``` unzip build.zip ``` You're now ready to start the http-server. Let the server run in the background ``` npm start ``` ## 1.3 Running the benchmarks and handling errors In a new console window you can now run the benchmarks: ``` npm run bench ``` This will take some time (currently about 12 hours on my machine). Finally create the results table: ``` npm run results ``` Open js-framework-benchmark/webdriver-ts-results/table.html in a browser and take a look at the results. You can open the result table with the link [http://localhost:8080/webdriver-ts-results/dist/index.html](http://localhost:8080/webdriver-ts-results/dist/index.html) Here's what you should do when the benchmark run was not successful. Let's assume the benchmark printed the following to the console: ``` ================================ The following benchmarks failed: ================================ Executing frameworks/non-keyed/ef-js and benchmark 04_select1k failed: No paint event found run was not completely successful Benchmarking failed with errors ``` You'll now have to run the benchmark again for those that failed like that: ``` npm run bench -- --framework non-keyed/ef-js --benchmark 04_ ``` The you can then continue with creating the results table `npm run results`. Another workaround is to delete the folders of frameworks you can't run or you are not interested in. # 2 The old and hard way: Building the frameworks and running the benchmark ## 2.1 Prerequisites Have _node.js (>=v16.14.2)_ installed. If you want to do yourself a favour use nvm for that and install yarn. The benchmark has been tested with node vv16.14.2. For some frameworks you'll also need _java_ (>=8, e.g. openjdk-8-jre on ubuntu). Please make sure that the following command work before trying to build: ``` > npm npm -version 8.5.0 > node --version v16.14.2 > echo %JAVA_HOME% / echo $JAVA_HOME > java -version java version "1.8.0_131" ... > javac -version javac 1.8.0_131 ``` ## 2.2 Start installing As stated above building and running the benchmarks for all frameworks can be challenging, thus we start step by step... Install global dependencies This installs just a few top level dependencies for the building the frameworks and a local web server. ``` npm ci ``` Then install the server: ``` npm run install-server ``` We start the local web server in the root directory ``` npm start ``` Verify that the local web server works: Try to open [http://localhost:8080/index.html](http://localhost:8080/index.html). If you see something like that you're on the right track:  Now open a new terminal window and keep the web server running in background. ## 2.3 Building and viewing a single framework We now try to build the first framework. Go to the vanillajs reference implementation ``` cd frameworks/keyed/vanillajs ``` and install the dependencies ``` npm ci ``` and build the framework ``` npm run build-prod ``` There should be no build errors and we can open the framework in the browser: [http://localhost:8080/frameworks/keyed/vanillajs/](http://localhost:8080/frameworks/keyed/vanillajs/) Some frameworks like binding.scala or ember can't be opened that way, because they need a 'dist' or 'target/web/stage' or something in the URL. You can find out the correct URL in the [index.html](http://localhost:8080/index.html) you've opened before or take a look whether there's a customURL property under js-framework-benchmark in the [package.json](https://github.com/krausest/js-framework-benchmark/blob/master/frameworks/keyed/ember/package.json#L10) that represents the url. ## 2.4 Running benchmarks for a single framework The benchmark uses an automated benchmark driver using chromedriver to measure the duration for each operation using chrome's timeline. Here are the steps to run is for a single framework: ``` cd ../../.. cd webdriver-ts ``` and install the dependencies ``` npm ci ``` and build the benchmark driver ``` npm run compile ``` now run the benchmark driver for the vanillajs-keyed framework: ``` npm run bench keyed/vanillajs ``` Just lean back and watch chrome run the benchmarks. If it doesn't complain then the html for the table should be fine and your categorization as keyed or non-keyed should also be correct. You should keep the chrome window visible since otherwise it seems like paint events can be skipped leading to wrong results. On the terminal will appear various log statements. The results for that run will be saved in the `webdriver-ts/results` directory. We can take a look at the results of a single result: ``` cat results/vanillajs-keyed_01_run1k.json {"framework":"vanillajs-keyed","benchmark":"01_run1k","type":"cpu","min":135.532,"max":154.821,"mean":143.79166666666666,"median":141.022,"geometricMean":143.56641695989177,"standardDeviation":8.114582360718808,"values":[154.821,135.532,141.022]} ``` As you can see the mean duration for create 1000 rows was 144 msecs. You can also check whether the implementation appears to be compliant to the rules: ``` npm run isKeyed keyed/vanillajs ``` If it finds anything it'll report an ERROR. ## 2.5 Building the result table Install libraries: ``` cd .. cd webdriver-ts-results npm ci cd .. cd webdriver-ts ``` In the webdriver-ts directory issue the following command: ``` npm run results ``` Now a result table should have been created which can be opened on [http://localhost:8080/webdriver-ts-results/dist/index.html](http://localhost:8080/webdriver-ts-results/dist/index.html). There's nothing in table except for the column vanillajs-keyed at the right end of the first table.  ## 2.6 [Optional] Updating the index.html file This simply rebuilds the file used to display the table, not the results. ``` npm run index ``` ## 2.7 [Optional] Building and running the benchmarks for all frameworks This is not for the faint at heart. **Please read the security advice before running this command.** You can build all frameworks by issuing: ``` cd .. npm run rebuild-all ``` After downloading the whole internet it starts building it. Basically there should be no errors during the build, but I can't guarantee that the dependencies won't break. You can now run the benchmark for all frameworks by invoking: ``` npm run bench-all ``` in the root directory. After that you can check all results in [http://localhost:8080/webdriver-ts/table.html](http://localhost:8080/webdriver-ts/table.html). # 3 Tips and tricks - You can run multiple implementations by passing their directory names (cd to webdriver-ts): `npm run bench keyed/angular keyed/react`. - You can select multiple frameworks and benchmarks for running with prefixes like in the following example in the webdriver-ts directory: `npm run bench -- --benchmark 01_ 02_ --framework keyed/vanillajs keyed/react-hooks` runs the test for all frameworks that contain either angular or bob, which means all angular versions and bobril and all benchmarks whose id contain 01* or 02* - The memory benchmarks assume certain paths for the chrome installation. If it doesn't fit use `npm run bench -- --chromeBinary /usr/bin/google-chrome` - If you can't get one framework to compile or run, just move it out of the frameworks directory and re-run - One can check whether an implementation is keyed or non-keyed via `npm run isKeyed` in the webdriver-ts directory. You can limit which frameworks to check in the same way as the webdriver test runner like e.g. `npm run isKeyed keyed/svelte`. The program will report an error if a benchmark implementation is incorrectly classified. ## 4. Contributing a new implementation ## 4.1 Example instructions for a real implementation Thanks @dsvorc41 for providing the following description: TL;DR:  1. Install all of the root-level dependencies 1. `cd js-framework-benchmark/` 1. `npm ci` or `npm i` 1. `npm run install-local` 2. Make a new directory for your desired framework, for example Fast framework: `mkdir /frameworks/keyed/fast` 3. Set up your new directory in whatever way is appropriate for that framework, for example: 1. Set up prettier, eslint, dependencies (i.e. `@microsoft/fast-element`) etc 2. Create `index.html` in the root of your folder where your app will be served `touch /frameworks/keyed/fast/index.html` 3. Note: your html file must use the global CSS styles `` 4. Serve the page - Test that your html page is loaded properly in the browser 1. For example put `