CP crawler

Summary

The perftracker-cp-crawler is a python wrapper for selenium, Chrome and Firefox to automate Control Panels performance testing (login, menu recognition, parallel load generation, etc)

Gallery

Gallery not found.

Installation

Option #1 – Installing from source, see https://github.com/perfguru87/perftracker-cp-crawler/blob/master/README.md

Option #2 – Installing the latest pre-built version from pypi.org

pip install perftracker-cp-crawler

Features set

  • selenium-based framework (on top of Chrome and Firefox browsers)
  • python-based browser simulators
  • export results to the perftracker server
  • browser memory consumption tracking
  • automatic login/logout with customizable login/password/login form xpath
  • automatic menu items recognition with customizsable:
    • menu item xpath
    • sub-menu item xpath
  • page rendering phases recognition:
    • browser timing interface support
    • ajax request completion based on browser logs and pending HTTP requests
    • HTTP requests whitelisting to bypass websockets and long polls completion wait
  • advanced reporting:
    • waterfall based requests view with information about requests size, compression, duration, status, etc
    • pages summary
    • HTML report with pages screenshot

Sources

https://github.com/perfguru87/perftracker-cp-crawler

Release Notes

  • 1.0.0 Jul 18, 2020
  • 0.0.44 Feb 6, 2019
    • change: rename –remote_connstring to –remote-connstring
  • 0.0.43 Feb 6, 2019
    • feature: Add –remote_connstring option to UI crawler to run the tests on the remote selenium grid
  • 0.0.42 Dec 6, 2018
    • bugfix: Fix cpus count and topology detection
  • 0.0.41 Dec 3, 2018
    • feature: pt-suite-uploader.py – support the perftacker job json format as input (-j option)
    • feature: perftrackerlib/client.py – added ptEnvNode.cpus_topology
    • change: change -j option behaviour: was ‘-f my.json -j’, now ‘-j my.json’
    • bugfix: encode HTTP response message if upload failed
    • bugfix: helpers/ptshell.py – fix CPUs calculation on Mac
  • 0.0.40 Nov 13, 2018
    • bugfix: fix ptEnvNode().hw_info.ram_kb type to int
  • 0.0.39 Nov 9, 2018
    • feature: add addGrafanaLink method for ptSuite
    • feature: ptSuite().addTest() – now it is possible to use addTest to add iterations (if tag, category and group matches)
  • 0.0.38 Oct 20, 2018
    • feature: pt_suite_example_ab.py – added –time option to limit tests by time
    • feature: pt_suite_example_ab.py – added -f option to parse requests from file
    • bugfix: add missed requirements in requirements.txt
    • bugfix: fix server API errors encoding
  • 0.0.37 Oct 10, 2018
    • feature: helpers/shell.py – new library for local/remote process execution and basic OS/HW info collection
    • feature: perftrackerlib/browser/cp_crawler.py – added browser name and CPU GHz when calcullate automatic job title
    • feature: examples/pt_suite_example_ab.py – new suite example based on ab (apache benchmark)
  • 0.0.36 Oct 7, 2018
    • feature: helpers/timeline.py – added a custom data_id parameter to ptTask
  • 0.0.35 Oct 6, 2018
    • feature: timeline.py – ptDoc() – added add_body() method to add custom html body
    • feature: timeline.py – ptTimeline() – added the ‘cluster’ parameter to disable events clustering on zoom out
    • bugfix: client.py – fix error handling on job upload
  • 0.0.34 Oct 4, 2018
    • feature: client.py – ptSuite() – enable autocompression for stdout/stderr output logs in the –pt-log-upload mode
    • feature: timeline/timeline.js – smooth item text redraw while user scrolls the items
  • 0.0.33 Oct 2, 2018
    • feature: improve autotests and code coveragate in test.py
    • feature: perftrackerlib/client.py – ptSuite now has –pt-log-upload and –pt-log-ttl options to upload stdout & stderr to the server and link it to the job
    • feature: pt-artifact-ctl.py – added ‘dump’ command support to download and print an artifact to stdout
    • bugfix: pt-artifact-ctl.py – fix artifact name in the ‘list’ mode
  • o.0.32 Oct 1, 2018
    • feature: artifacts management (tools/pt-artifact-ctl.py, and ptArtifact class in the client.py)
  • 0.0.31 Sep 22, 2018
    • bugfix: browser/page.py – fix page size in the summary (print in KBytes, not in Bytes)
    • bugfix: browser/cp_engine.py – strip menu item title
  • 0.0.30 Sep 07, 2018
    • perftracker-lib.spec – fix problem with /bin/pt-suite-uploader.py removal after ‘yum update’ or ‘rpm -ihv’. You have to delete (rpm -e) and then install (rpm -ihv) new version (>= 0.0.30) to resolve the issue 
  • 0.0.29 Sep 04, 2018
    • examples/pt-suite-uploader.py – fix parsing of ‘less_better: false’
  • 0.0.28 Aug 16, 2018
    • perftracker-lib.spec – added libjpeg-devel, zlib-devel requirements needed for Pillow installation
  • 0.0.27 Aug 16, 2018
    • browser/cp_crawler.py – fix broken compatibility in crawl() return value
    • perftrackerlib/client.py
      • added project name validation on first upload
      • do not hardcode default project name anymore
      • fix ‘addOptions(pt_url=…) is deprecated’ message if crawler override pt portal URL
      • fix json serialization on old python2
  • 0.0.26 Aug 15, 2018
    • perftrackerlib/client.py – automatically add http:// prefix if not specified in the –pt-url command line
    • perftrackerlib/client.py – fix –pt-product-name option handler
  • 0.0.25 Aug 15, 2018
    • perftrackerlib/client.py – fix serialization to json for utf-8 members
    • perftracker-lib.spec – remove /bin/pt-suite-uploader.py on uninstall
  • 0.0.24 Aug 15, 2018
    • deploy pt-suite-uploader.py to /bin (or python home bin/ dir path)
  • 0.0.23 Aug 14, 2018
    • examples/pt-suite-uploader.py – a tool to parse tests results directly from text/json and upload to the server
  • 0.0.22 Aug 13, 2018
    • perftrackerlib/client.py – fix missed fractional part of the scores below 100
  • 0.0.21 Aug 12, 2018
    • perftrackerlib/client.py – fix exception in –pt-project option
  • 0.0.20 Aug 7, 2018
    • examples/
      • new pt_suite_example_minimal.py
    • client.py
      • set correct defaults and make command line arguments optional
  • 0.0.19 Aug 7, 2018
    • installation
      • added MANIFEST.in to fix installation from pypi.org (missed requirements.txt)
  • 0.0.18 Aug 7, 2018
    • examples/pt-wp-crawler.py
      • fix exceptions in cp_skip_menu_item()
    • browser/
      • clear the user/password form field if it is not empty
      • fix page title in HTML report
      • optimize uncached pages retrival a bit since now this is frequent operation
      • use -g option to generate HTML report with pages details
      • fix problem with wrong link assignment to a menu item
      • fix cp_init_context() before the first login attempt
      • added sbmt_xpath parameter to specify submit button xpath
    • helpers/timeline.py
      • added the ‘groups_title’ parameter
      • timeline.js – set min group height to 25px
      • timeline.css – improve timeline event alignment
      • fix JS deprication warning
      • ptTimeline ‘title’ is now optional
      • timeline.js – fix initial range if ‘start’ and ‘end’ was passed to the Timeline constructor
    • installation
      • added missed *.js and *.css files to the list of installed artefacts
      • added perftracker-lib.spec
      • requirements.txt – exclude psycopg2
  • 0.0.17 Jul 18, 2018
    • let it be initial version…

ToDo

…