Pytest automate

.gitignore

@@ -0,0 +1,5 @@
+.vscode
+__pycache__
+.pytest_cache
+env
+local.log

README.md

@@ -1,7 +1,137 @@
-# BrowserStack Smart Test Selection AI Agent
+![BrowserStack Logo](https://d98b8t1nnulk5.cloudfront.net/production/images/layout/logo-header.png?1469004780)
+
+# Smart Test Selection & Orchestration AI Agent Demo for BrowserStack Automate
+
+## Table of Contents
+- [What is Smart Test Selection?](#what-is-smart-test-selection)
+
+- [How Smart Test Selection Agent Works](#how-smart-test-selection-agent-works)
+
+- [Steps to run the demo](#steps-to-run-the-demo)
+
 ---
+
 ## What is Smart Test Selection?
-BrowserStack's Smart Test Selection Agent uses AI to identify and run only the tests impacted (or likely to fail) by your code changes, reducing build time and cost by up to 50%. This demo uses the BrowserStack Java SDK with TestNG.
+BrowserStack's Smart Test Selection Agent uses AI to identify and run only the tests impacted by your code changes, reducing build time and cost. This demo uses the BrowserStack Python SDK with PyTest: 
+- App Repo – [test-selection-demo-app-browserstack](https://github.com/browserstack/test-selection-demo-app-browserstack)
+- Test Repo – [test-selection-demo-test-browserstack](https://github.com/browserstack/test-selection-demo-test-browserstack)
+
+---
+
+## How Smart Test Selection Agent Works
+- Agent does a static analysis of the test repo to understand the tests structure, workflows being tested etc. Based on the code changes and static analysis done earlier, it then predicts impacted tests for incoming builds.
+- Agent needs read-only access to the application repo and test repo - this can be given via BrowserStack Github app
+
+For detailed integration steps follow the [Test Selection Documentation](https://www.browserstack.com/docs/automate/selenium/smart-test-selection).
+
+## Steps to run the demo
+- This demo showcases BrowserStack's Impact Analysis agent using the **Github App** integration approach.
+- First, we run a build with Impact Analysis disabled and then run the same build with it enabled, allowing you to clearly see the impact - with Impact Analysis, only the tests impacted by the code change run, resulting in a reduction in execution time.
+
+### Prerequisites
+- Python 3.9+
+- BrowserStack Account with AI Enabled: [Activate BrowserStack AI preferences](https://www.browserstack.com/docs/iaam/settings-and-permissions/activate-browserstack-ai)
+
+### Step 1: Setup Test Repo
+
+```
+# Clone the test repo
+git clone https://github.com/browserstack/test-selection-demo-test-browserstack.git
+cd test-selection-demo-test-browserstack
+
+> If your org is not `browserstack`, fork the test and app repos into your own org first, and clone the forked test repo instead.
+
+# Checkout the pytest demo branch
+git checkout pytest-automate
+
+# Create and activate a virtual environment
+python3 -m venv env
+source env/bin/activate   # on Mac
+env\Scripts\activate      # on Windows
+
+# Install dependencies
+pip install -r requirements.txt
+```
+
+### Step 2: Add BrowserStack Credentials
+- Update `username` and `accessKey` in the `browserstack.yml` file with your BrowserStack access credentials found [here](https://www.browserstack.com/accounts/profile/details)
+```
+userName: <your-browserstack-username>
+accessKey: <your-browserstack-accesskey>
+```
+
+### Step 3: Connect your repos for Impact Analysis
+- Provide read-only access to the demo test and app repos via [BrowserStack GitHub Actions](https://automate.browserstack.com/integrations?category=cicd) in Integrations (user with BrowserStack Admin + Github Admin access can configure the repos)
+- In the AI Agents Configuration page under Settings in BrowserStack, add the details for the test repo (e.g. `pytest-automate` branch, `tests` folder) along with the app repo.
+A sample configuration is shown in the image below
+![Test Selection Repo Configuration](./test-selection-repo-config.png)  
+
+- Update the app repo reference used for impact mapping:
+  Create a test-selection.json file in the demo test repo. Specify the feature branch of the app repo (which has the code changes). A sample is shown below 
+```json
+{
+  "DEMO_APP": {
+    "url": "https://github.com/<YOUR_ORG>/test-selection-demo-app-browserstack",
+    "featureBranch": "demo-temp"
+  }
+}
+```
+  Replace `<YOUR_ORG>` with your own GitHub org (not `browserstack`).
+- Once configured, trigger static analysis of the test repo by clicking on 'Analyse Test Repo' in AI Agent Configuration settings — this is required before running a build with Impact Analysis enabled, and takes 5-10 minutes.
+
+### Step 4: Run a build without Impact Analysis enabled
+- In `browserstack.yml`, add projectName, buildName and disable Test Selection
+```yaml
+projectName: Smart Test Selection Impact Analysis Demo Project
+buildName: Smart Test Selection Impact Analysis Demo Build
+
+testOrchestrationOptions:
+  runSmartSelection:
+    enabled: false
+    mode: 'relevantOnly'
+```
+
+- Run the build with:
+```
+browserstack-sdk pytest tests/
+```
+- Results:
+  - All tests in the suite run
+  - This is our baseline - the full suite always runs regardless of which parts of the code changed
+
+### Step 5: Run a build with Impact Analysis enabled
+- In `browserstack.yml`, enable Test Selection
+```yaml
+projectName: Smart Test Selection Impact Analysis Demo Project
+buildName: Smart Test Selection Impact Analysis Demo Build
+
+testOrchestrationOptions:
+  runSmartSelection:
+    enabled: enable
+    mode: 'relevantOnly'
+```
+
+- Run the build with:
+```
+browserstack-sdk pytest tests/
+```
+- Results:
+  - Impact Analysis analyzes the code changes in the demo app
+  - It determines which tests are impacted by those changes
+  - Instead of running the full suite, the agent selects and runs only the impacted tests
+  - All other unimpacted tests are automatically skipped/dropped
+---
+
+### Step 6: View Test Selection Report
+- Once the build with Impact Analysis enabled completes, open the build on the BrowserStack Automate dashboard and go to the **Test Selection Report** in the AI Agents Report tab.
+- The report shows:
+  - The list of tests selected to run for this build
+  - The reasoning behind why the agent selected each test 
+  - The tests that were skipped, since they weren't impacted by the code change
+
+![Test Selection Report](./test-selection-report.png)  
+
+---
 
-## Introduction
-This repository demonstrates BrowserStack's Smart Selection AI Agent. For demo steps, checkout branch [`tesng-automate`](https://github.com/browserstack/test-selection-demo-test-browserstack/tree/testng-automate) and follow the readme
+## Notes
+* You can view your test results on the [BrowserStack Automate dashboard](https://www.browserstack.com/automate)

browserstack.yml

@@ -0,0 +1,26 @@
+userName: username
+accessKey: password
+
+projectName: Smart Test Selection Impact Analysis Demo Project
+buildName: Smart Test Selection Impact Analysis Demo Build 
+buildIdentifier: '#${BUILD_NUMBER}'
+framework: pytest
+
+testObservability: true
+automation: false
+browserstackLocal: false
+platforms:
+  - os: Windows
+    osVersion: 11
+    browserName: Chrome
+    browserVersion: latest
+
+source: pytest-browserstack:local-sdk:v1.0
+debug: true
+logLevel: debug
+
+testOrchestrationOptions:
+  runSmartSelection:
+    enabled: true
+    mode: 'relevantFirst'
+    source: 'test-selection.json'

requirements.txt

@@ -0,0 +1,8 @@
+browserstack-local
+selenium>=3.14
+pytest==7.4.4
+pytest-selenium
+pyobjc-core<12 ; sys_platform == "darwin"
+pyobjc-framework-Cocoa<12 ; sys_platform == "darwin"
+pyobjc-framework-SystemConfiguration<12 ; sys_platform == "darwin"
+browserstack-sdk @ https://sdk-assets.browserstack.com/python/browserstack_sdk-latest.tar.gz

test-selection.json

@@ -0,0 +1,6 @@
+{
+  "DEMO_APP": {
+    "url": "https://github.com/browserstack/test-selection-demo-app-browserstack",
+    "featureBranch": "demo-temp"
+  }
+}

tests/auth/test_login_invalid_credentials.py

@@ -0,0 +1,17 @@
+import time
+
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+
+
+def test_login_fails_with_invalid_credentials(demo):
+    demo.open_login_page()
+    demo.driver.find_element(By.ID, "email-input").send_keys("fake@example.com")
+    demo.driver.find_element(By.ID, "password-input").send_keys("wrong-password")
+    time.sleep(1)
+    demo.driver.find_element(By.ID, "login-submit").click()
+    error_banner = demo.wait_for(5).until(
+        EC.visibility_of_element_located((By.ID, "login-error")))
+    assert "Login failed" in error_banner.text, \
+        "Inline login error banner should mention login failure"
+    assert "/login" in demo.driver.current_url

tests/auth/test_login_password_toggle.py

@@ -0,0 +1,9 @@
+from selenium.webdriver.common.by import By
+
+
+def test_show_password_toggle_changes_input_type(demo):
+    demo.open_login_page()
+    password_field = demo.driver.find_element(By.ID, "password-input")
+    assert password_field.get_attribute("type") == "password"
+    demo.driver.find_element(By.ID, "show-password").click()
+    assert password_field.get_attribute("type") == "text"

tests/auth/test_login_quick_select.py

@@ -0,0 +1,13 @@
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+
+
+def test_quick_select_user_can_login(demo):
+    email = demo.first_demo_user_email()
+    demo.login_using_quick_select(email)
+    demo.open_profile_page()
+    demo.wait_for(10).until(EC.visibility_of_element_located(
+        (By.XPATH, "//h2[contains(@class,'text-xl')]")))
+    profile_email = demo.driver.find_element(
+        By.XPATH, "//div[contains(@class,'text-gray-600')][1]")
+    assert email in profile_email.text

tests/auth/test_login_redirect_when_authenticated.py

@@ -0,0 +1,8 @@
+from selenium.webdriver.support import expected_conditions as EC
+
+
+def test_visiting_login_while_authenticated_redirects_to_profile(demo):
+    demo.login_with_random_demo_user()
+    demo.open_login_page()
+    demo.wait_for(5).until(EC.url_contains("/profile"))
+    assert demo.driver.current_url.endswith("/profile")

tests/auth/test_logout_clears_session.py

@@ -0,0 +1,12 @@
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+
+
+def test_logout_from_profile_clears_session(demo):
+    demo.login_with_random_demo_user()
+    demo.open_profile_page()
+    demo.wait_for(5).until(EC.visibility_of_element_located(
+        (By.XPATH, "//button[contains(text(),'Logout')]"))).click()
+    login_header = demo.wait_for(5).until(EC.visibility_of_element_located(
+        (By.XPATH, "//h1[contains(text(),'Login') or contains(text(),'Welcome Back')]")))
+    assert "Login" in login_header.text or "Welcome" in login_header.text

tests/bstack-local-test.py

@@ -0,0 +1,6 @@
+import pytest
+
+def test_local(selenium):
+  selenium.get('http://bs-local.com:45454')
+  local_page_title = selenium.title
+  assert local_page_title == 'BrowserStack Local'

tests/bstack-sample-test.py

@@ -0,0 +1,20 @@
+import pytest
+from selenium.webdriver.common.by import By
+
+def test_example(selenium):
+    selenium.get('https://bstackdemo.com/')
+
+    # locating product on webpage and getting name of the product
+    productText = selenium.find_element(By.XPATH, '//*[@id="1"]/p').text
+
+    # clicking the 'Add to cart' button
+    selenium.find_element(By.XPATH, '//*[@id="1"]/div[4]').click()
+
+    # waiting until the Cart pane has been displayed on the webpage
+    selenium.find_element(By.CLASS_NAME, 'float-cart__content')
+
+    # locating product in cart and getting name of the product in cart
+    productCartText = selenium.find_element(By.XPATH, '//*[@id="__next"]/div/div/div[2]/div[2]/div[2]/div/div[3]/p[1]').text
+
+    # checking whether product has been added to cart by comparing product name
+    assert productCartText == productText

tests/cart/test_cart_badge_default.py

@@ -0,0 +1,8 @@
+from selenium.webdriver.common.by import By
+
+
+def test_cart_badge_starts_at_zero(demo):
+    demo.go_home()
+    badge_value = demo.driver.find_element(
+        By.CSS_SELECTOR, "#shopping-cart-btn span").text.strip()
+    assert badge_value == "0"

tests/cart/test_cart_checkout_visibility.py

@@ -0,0 +1,10 @@
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+
+
+def test_checkout_button_hidden_when_logged_out(demo):
+    demo.add_product_to_cart(6)
+    demo.open_cart_page()
+    notice = demo.wait_for(5).until(EC.visibility_of_element_located(
+        (By.XPATH, "//div[contains(@class,'text-yellow-800')]")))
+    assert "Please" in notice.text

tests/cart/test_cart_clear_all.py

@@ -0,0 +1,13 @@
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+
+
+def test_clear_cart_button_removes_all_items(demo):
+    demo.add_products_to_cart(1, 2)
+    demo.open_cart_page()
+    clear_button = demo.wait_for(5).until(
+        EC.element_to_be_clickable((By.ID, "clear-cart-btn")))
+    clear_button.click()
+    empty_message = demo.wait_for(5).until(EC.visibility_of_element_located(
+        (By.XPATH, "//p[text()='Your cart is empty.']")))
+    assert empty_message.is_displayed()

tests/cart/test_cart_displays_items.py

@@ -0,0 +1,10 @@
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+
+
+def test_cart_displays_items_added_from_catalog(demo):
+    demo.add_product_to_cart(4)
+    demo.open_cart_page()
+    name = demo.wait_for(5).until(EC.visibility_of_element_located(
+        (By.XPATH, "//div[@id='cart-item']//div[contains(@class,'font-semibold')]")))
+    assert "Galaxy Note" in name.text

tests/cart/test_cart_empty_state.py

@@ -0,0 +1,10 @@
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+
+
+def test_cart_shows_empty_state_for_empty_cart(demo):
+    demo.login_with_random_demo_user()
+    demo.open_cart_page()
+    empty = demo.wait_for(5).until(EC.visibility_of_element_located(
+        (By.XPATH, "//p[contains(text(),'Your cart is empty.')]")))
+    assert empty.is_displayed()

tests/cart/test_cart_quantity_adjustment.py

@@ -0,0 +1,17 @@
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+
+
+def test_increment_and_decrement_buttons_update_quantity(demo):
+    demo.add_product_to_cart(5)
+    demo.open_cart_page()
+    quantity_locator = (
+        By.XPATH, "//div[@id='cart-item']//span[contains(@id,'item-quantity')]")
+    plus_button = demo.driver.find_element(
+        By.XPATH, "//div[@id='cart-item']//button[text()='+']")
+    minus_button = demo.driver.find_element(
+        By.XPATH, "//div[@id='cart-item']//button[text()='-']")
+    plus_button.click()
+    demo.wait_for(5).until(EC.text_to_be_present_in_element(quantity_locator, "2"))
+    minus_button.click()
+    demo.wait_for(5).until(EC.text_to_be_present_in_element(quantity_locator, "1"))

tests/cart/test_cart_remove_item.py

@@ -0,0 +1,12 @@
+from selenium.webdriver.common.by import By
+
+
+def test_remove_item_link_removes_single_item(demo):
+    demo.add_products_to_cart(7, 8)
+    demo.open_cart_page()
+    before_items = demo.driver.find_elements(By.ID, "cart-item")
+    demo.driver.find_element(
+        By.XPATH, "(//div[@id='cart-item']//button[contains(text(),'Remove')])[1]").click()
+    demo.wait_for(5).until(
+        lambda d: len(d.find_elements(By.ID, "cart-item")) == len(before_items) - 1
+    )

tests/checkout/test_checkout_invoice_modal.py

@@ -0,0 +1,21 @@
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+
+
+def test_invoice_modal_displays_line_items(demo):
+    demo.login_with_random_demo_user()
+    demo.add_product_to_cart(2)
+    demo.open_cart_page()
+    demo.driver.find_element(By.ID, "checkout-btn").click()
+    demo.wait_for(5).until(
+        EC.element_to_be_clickable((By.ID, "place-order-btn"))).click()
+    demo.wait_for(10).until(EC.url_contains("/orders"))
+    first_order = demo.wait_for(5).until(
+        EC.visibility_of_element_located((By.CSS_SELECTOR, "li.order-list-item")))
+    order_id = first_order.get_attribute("data-order-id")
+    demo.driver.find_element(By.ID, f"view-invoice-btn-{order_id}").click()
+    modal_header = demo.wait_for(5).until(EC.visibility_of_element_located(
+        (By.XPATH, "//h2[contains(text(),'Invoice for Order')]")))
+    assert order_id in modal_header.text
+    assert demo.driver.find_elements(
+        By.XPATH, "//div[contains(@class,'shadow-lg')]//li") != []