Attachment 'TestIdeas.txt'

Download

   1 Tests for cloud images
   2 ======================
   3 
   4 0. Check the generate image is within reasonable size bounds of previous image (eg +/- 20%)
   5 1. Image boots on provider X - each provider (within a reasonable time)
   6 2. We can login with username we specified
   7 3. We can install some packages
   8 4. If we installed something, we should be able to use this software.
   9 5. Test that changes are followed (e.g. no login for root user)
  10 6. Tests for cloud-specific settings (e.g. disabling IPv6)?
  11 7. Log files does not contain errors, creation logs do not contain errors
  12 8. Size of image should be similar to size of previous image. Similarly for the logs. If they are 2x bigger, you should read it.
  13 9. No unexpected open ports
  14 10. sources.list as we expect (only debian)
  15 11. check we don't have agents for other platforms in a specific platform image
  16 12. check that ssh root password auth is disabled
  17 13. No unexpected users
  18 14. check cloud-init startup (if configured)
  19 15. Check that /tmp/, /var/tmp is writeable (echo >/tmp/foo, cat /tmp/foo, rm /tmp/foo)
  20 16. Check filesystem is read/write (Manoj)
  21 17. Check correct network driver is in use (eg, EC2 SR IOV or EC2 ENA)
  22 18  Test /var/log/ directory? (should be empty of build-instance logs)
  23 19. If we provide ability to switch languages (locales), we need to test it.
  24 
  25 
  26 Test framework requirements (what's already in the archive?)
  27 ===========================
  28 1. Must be able to launch instances of a given image
  29 2. Must be able to perform local operations: e.g. verify that the instance transitions into a "running" state according to cloud API, is reachable on the network, etc.
  30 3. Must allow developers to easily create new tests
  31 4. Must support a common set of tests, plus provider-specific tests
  32 5. Must be able to execute multiple test runs with various different instance launch parameters
  33 6. Must produce useful logs that can be compared across runs.
  34 7. Should allow downstream image builders, not just developers, to easily add tests for potentially custom images

Attached Files

To refer to attachments on a page, use attachment:filename, as shown below in the list of files. Do NOT use the URL of the [get] link, since this is subject to change and can break easily.
  • [get | view] (2016-11-08 00:05:57, 2.8 KB) [[attachment:CloudFrontProxyInterceptionConfig.txt]]
  • [get | view] (2016-11-08 00:05:00, 41.4 KB) [[attachment:Minutes.txt]]
  • [get | view] (2016-11-08 00:05:32, 2.0 KB) [[attachment:TestIdeas.txt]]
  • [get | view] (2016-11-30 15:11:10, 272.0 KB) [[attachment:group_photo.jpg]]
 All files | Selected Files: delete move to page copy to page

You are not allowed to attach a file to this page.