test/py: Implement pytest infrastructure

This tool aims to test U-Boot by executing U-Boot shell commands using the
console interface. A single top-level script exists to execute or attach
to the U-Boot console, run the entire script of tests against it, and
summarize the results. Advantages of this approach are:

- Testing is performed in the same way a user or script would interact
  with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself.
  It is asserted that writing test-related code in Python is simpler and
  more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.

A few simple tests are provided as examples. Soon, we should convert as
many as possible of the other tests in test/* and test/cmd_ut.c too.

The hook scripts, relay control utilities, and udev rules I use for my
own HW setup are published at https://github.com/swarren/uboot-test-hooks.

See README.md for more details!

Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Michal Simek <michal.simek@xilinx.com>
Tested-by: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org> #v3

3 files changed, 43 insertions, 0 deletions

diff --git a/test/py/tests/test_000_version.py b/test/py/tests/test_000_version.py
new file mode 100644
index 0000000..d262f05
--- /dev/null
+++ b/test/py/tests/test_000_version.py
@@ -0,0 +1,20 @@
+# Copyright (c) 2015 Stephen Warren
+# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+# pytest runs tests the order of their module path, which is related to the
+# filename containing the test. This file is named such that it is sorted
+# first, simply as a very basic sanity check of the functionality of the U-Boot
+# command prompt.
+
+def test_version(u_boot_console):
+    '''Test that the "version" command prints the U-Boot version.'''
+
+    # "version" prints the U-Boot sign-on message. This is usually considered
+    # an error, so that any unexpected reboot causes an error. Here, this
+    # error detection is disabled since the sign-on message is expected.
+    with u_boot_console.disable_check('main_signon'):
+        response = u_boot_console.run_command('version')
+    # Ensure "version" printed what we expected.
+    u_boot_console.validate_version_string_in_text(response)
diff --git a/test/py/tests/test_help.py b/test/py/tests/test_help.py
new file mode 100644
index 0000000..894f3b5
--- /dev/null
+++ b/test/py/tests/test_help.py
@@ -0,0 +1,9 @@
+# Copyright (c) 2015 Stephen Warren
+# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+def test_help(u_boot_console):
+    '''Test that the "help" command can be executed.'''
+
+    u_boot_console.run_command('help')
diff --git a/test/py/tests/test_unknown_cmd.py b/test/py/tests/test_unknown_cmd.py
new file mode 100644
index 0000000..2de93e0
--- /dev/null
+++ b/test/py/tests/test_unknown_cmd.py
@@ -0,0 +1,14 @@
+# Copyright (c) 2015 Stephen Warren
+# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+def test_unknown_command(u_boot_console):
+    '''Test that executing an unknown command causes U-Boot to print an
+    error.'''
+
+    # The "unknown command" error is actively expected here,
+    # so error detection for it is disabled.
+    with u_boot_console.disable_check('unknown_command'):
+        response = u_boot_console.run_command('non_existent_cmd')
+    assert('Unknown command \'non_existent_cmd\' - try \'help\'' in response)