Gallery

Gallery not found.

Installation

See https://github.com/perfguru87/perftracker/blob/master/README.md

Features set

Tests
- test scores, deviations, command line, description
- number of errors, warnings
- test start, end, duration
- link to Grafana, custom HTTP links
Jobs
- REST API for Job results upload
- UI for manual job results download/upload
- Job details – summary, tests list and tests details
Comparisons
- correct results comparison (considering deviations)
- line and bar charts, tables
- note: comparisons are manual only at the moment
Regression tracking
- automatic build-to-build comparison according to job regression tag
- automatic recognition of test cases with:
  - degradations > 50%
  - degradation > 10%
  - improvement > 10%
  - improvement > 50%
Artifacts management
- a test or a job may have custom artifacts such as logs or data uploaded to the server
Hosts management
- Hosts list
- Hosts details – os, title, ip, hostname, num of CPUs, RAM size, HDD details, Network details, etc
- Host locking (manual or through API) with customizable lock types

Sources

https://github.com/perfguru87/perftracker

Release Notes

0.4.4 Dec 3, 2018
- Features:
  - comparison view – add test description when clicking on test details
  - comparison view – improve chart line smoothness
  - job upload – test and job uuids now are optional
  - job upload – skip empty tests on job results upload
  - hw env details – show cpus_topology property, e.g (2S x 4C x 2T) means 2 sockets, 4 cores, 2 hyper-threads
- Bugfixes:
  - fix comparison charts
  - comparison view – fix group section collapse/expand
0.4.3 Nov 4, 2018
- Features:
  - added test endpoint @ /test_endpoint/ to explain the performance CI workflow example in the documentation
  - add the ‘Project’ column in regression/comparison/job tables in ‘All projects’ mode
- Bugfixes:
  - fix comparisons, jobs and regressions list in the ‘All projects’ mode
  - jobs view – fix the link to the job details which was recently broken
  - models/test.py – increase storage for the ‘scores’ string of a test
  - regression details – fix links to the jobs
  - models/comparison.py – fix DivisionByZero in case of an average score is zero
  - comparisons view – fix the link to comparisons
  - comparison view – fix an issue on the chart when job A has test1 score and job B has only test2 score
0.4.2 Oct 10, 2018
- Features:
  - Hardware management – implement search hw nodes by node name, os, hostname, purpose and model
  - Regressions – show the total number of jobs in the regression in the header
  - Jobs upload – allow to upload a job w/o the tests, useful for just started jobs
- Bugfixes:
  - fix manual job results upload
  - fix artifacts uploading and download
  - fix node locks management in the admin panel
  - fix job json upload for jobs with hardware nodes info
  - make the ‘purpose’ field optional when edit/create it in the admin panel
0.4.1 Oct 7, 2018
- Hardware management:
  - feature: show lock edit dialog when clicking on the lock in the locks timeline
  - feature: hardware nodes now have the ‘purpose’ column
0.4.0 Oct 6, 2018
- Hardware management:
  - feature: added custom lock types (name, background, foreground)
  - feature: align lock name to the left side of the viewport and make it always visible
  - bugfix: pre-select given node when creating a lock
  - bugfix: disable locks clustering
  - bugfix: fix the issue with multiple lock acquisition
  - bugfix: fix the link to hardware edit form
  - bugfix: fix an issue when Unlock button is disabled after a user opens the lock creation dialog and cancels it
- Jobs comparison:
  - bugfix: fix jobs order in the comparison
- Other:
  - bugfix: fix server start on Windows
  - bugifx: added missed psycopg2 requirement to be able to start perftracker on postgres
0.3.1 Oct 2, 2018
- Jobs view:
  - feature: show the job artifacts on the job details panel
0.3.0 Oct 1, 2018
- Artifacts management:
  - feature: new ‘artifacts’ management API: upload, download, update, info, list, delete, link/unlink a test/job
- Results upload:
  - bugfix: do not allow to upload jobs with the duplicated combination of the ‘group’, ‘tag’ and ‘category’ attributes of the tests
- Comparison view:
  - bugfix: fix test details view of the jobs with multiple identical tests (group, tag, category). Such suites and jobs were allow
    ed before Perftracker 3.0
- Regressions view:
  - feature: added a link to the job on the regression page
  - feature: added ability to include/exclude jobs in regression
  - bugfix: determine ‘less_better’ regression attribute based on the last test in the last job
  - bugfix: skip jobs from another project (even if regression tag matches)
- Tests view:
  - feature: show test group on the job tests list page
  - bugfix: fix test duration (sec) in the tests list
- Jobs view:
  - feature: implement job removal
  - feature: show the product name and product version on the jobs list page
  - feature: move job link to the ‘id’ column, not the ‘Run title’ column (because a title can be empty)
  - bugfix: fix column titles
- Hardware view:
  - bugfix: fix hardware nodes list for all projects (project_id=0)
  - bugfix: fix nodes selection when creating a new lock on the hardware farm page
  - bugfix: filter out the hosts which are locked already when trying to acquire a new lock
0.2.2 Aug 17, 2018
- fixed the job ‘append’ mode:
  - do not delete existing tests results if job already exists
  - delete prev tests results if a job was marked as ‘deleted’ before
0.2.1 | Aug 16, 2018
- /api/v1.0/0/project – new API to fetch known projects
- POST /api/v1.0/0/job – put the list of existing projects to the error message if project_name in json doesn’t exist
0.2.0 | Aug 13, 2018
- added UI for job results download/upload
0.1.0 | Aug 7, 2018
- Let it be the initial version…

ToDo

Settings management
- which tabs to show
- host locking on/off
- custom tabs/pages
- default columns
Authentication / Authorization
- sign in form
- do not allow to release foreign hardware locks
- hide jobs/reports from anonymous users
Jobs list / details
- Highlight the jobs in progress
- Highlight the test in progress
- Download as XLS
- Download as JSON – done
- Upload button – done
Comparisons
- ‘Preview’ button
- Comparison edit dialog
- Comparison delete
- Bugfix – ‘back’ button doesn’t work after creating the comparison
- Better autocolors
- Ability to quickly hide/show/change charts
- Ability to quickly hide/show/change table
- Table columns specifier (deviation, test time, etc)
Regressions
- Automatically create/update comparison of 2 first vs 2 last jobs
- Mark regression as ‘investigated‘
- Email notifications about new degradations/improvements
- JIRA items lifecycle on the regression chart
- Detailed regression chart
- A quick comparison of selected builds
- Automatically create new regression if doesn’t exist
- Exclude job runs from regressions
- Exclude individual tests from regressions
Test results
- custom blobs upload and view in test details
- Error/warning messages
- Percentiles charts
- 3D charts (2 sets of categories)
Hosts view
- Change SSH login message on a host when it is locked
- Small CPU/RAM usage chart from Grafana (svg+polyline)
Custom links page
Custom pages
- iframe
- page rendered as some arbitrary command line output
JIRA page
- Additional fields for JIRA items
- JIRA items tree view