This document outlines the code generation step for the "Unit Test Generator" workflow. As part of our commitment to delivering high-quality, maintainable software, this step focuses on automatically generating comprehensive unit tests for given source code. This significantly accelerates the development process, enhances code quality, and ensures robust application behavior.
The Unit Test Generator is designed to automate the creation of foundational unit tests for Python functions. By analyzing the provided function's signature and (optionally) its docstrings or inferred behavior, it constructs a boilerplate unittest test suite, including common test cases and assertion patterns. This tool aims to reduce the manual effort of writing repetitive test structures, allowing developers to focus on more complex, business-logic-specific test scenarios.
A Unit Test Generator is an automated system that takes a piece of source code (e.g., a function, method, or class) as input and produces corresponding unit test code. It leverages patterns, heuristics, or advanced analysis (like LLMs in preceding steps) to infer potential test cases, edge cases, and expected behaviors.
unittest Framework: Generates tests compatible with Python's standard unittest module.The strategy for generating unit tests involves the following steps:
gemini step) that understands the function's intent and provides specific test case suggestions. For this deliverable, we use simple heuristics.unittest.TestCase subclass.test_scenario_name) within the test class.self.assertEqual, self.assertRaises, or other self.assert* calls.Below is a Python implementation of a UnitTestGenerator class. It takes a Python function's source code as input and generates a corresponding unittest file.
Let's consider a simple calculate_discount function for which we want to generate unit tests:
#### 4.2 Unit Test Generator Code (`unit_test_generator.py`) This code defines the `UnitTestGenerator` class that will produce the tests.
This document outlines a comprehensive and detailed study plan for developing a "Unit Test Generator." This deliverable represents Step 1 of 3: gemini → plan_architecture for the "Unit Test Generator" workflow, focusing on establishing the foundational learning path and project structure.
This plan details a structured approach to acquiring the necessary knowledge and skills to design, develop, and implement a robust "Unit Test Generator." It is tailored for individuals seeking to automate test creation, deepen their understanding of unit testing principles, code analysis, and test automation.
As part of the PantheraHive workflow, this deliverable ensures a clear roadmap for the subsequent development phases of the "Unit Test Generator" project.
To empower the learner with the expertise to conceptualize, build, and deploy a functional "Unit Test Generator" tool. This includes understanding the theoretical underpinnings of unit testing, mastering code introspection techniques, and applying automated code generation principles to produce high-quality, maintainable unit tests for a chosen programming language and framework.
* Proficiency in at least one object-oriented programming language (e.g., Python, Java, C#, JavaScript/TypeScript).
* Familiarity with fundamental software engineering concepts (e.g., functions, classes, data structures, algorithms).
* Basic understanding of unit testing concepts (e.g., assertions, test fixtures, mocks, stubs).
* Working knowledge of a version control system (e.g., Git).
This intensive study plan is designed for 6 weeks, assuming a commitment of approximately 10-15 hours of study and practical work per week.
Upon successful completion of this study plan, the learner will be able to:
* Introduction to Unit Testing: Purpose, benefits, types of tests (unit, integration, E2E), TDD vs. BDD.
* Key concepts: Assertions, test fixtures, test runners.
* Selection of target language and testing framework for the generator (e.g., Python/pytest, Java/JUnit, C#/NUnit, TypeScript/Jest).
* Introduction to code parsing: Abstract Syntax Trees (AST) and reflection mechanisms.
* Basic AST traversal and node inspection (e.g., identifying function definitions).
* Extensive readings on unit testing best practices and framework documentation.
* Hands-on exploration of the AST module/reflection API in the chosen language.
* Write small scripts to parse a simple source file and extract basic function/method names and their direct parameters.
* Advanced AST traversal: Handling classes, methods within classes, return types, decorators, and import statements.
* Identifying internal and external dependencies within code.
* Principles of effective test case design: Equivalence partitioning, boundary value analysis, error guessing, state-based testing.
* Introduction to mocking and stubbing frameworks: Why and how they are used.
* Enhance parsing scripts to extract information from more complex code structures (e.g., classes with multiple methods, method return types).
* Manually design comprehensive test cases (inputs, expected outputs, edge cases) for several moderately complex functions/methods.
* Research and experiment with a mocking library in the chosen language (e.g., unittest.mock, Mockito, Sinon.js).
* Structuring generated test files and classes according to framework conventions.
* Automating the generation of basic test method signatures (e.g., test_function_name).
* Logic for generating common assertions based on function return types and simple expected outcomes.
* Strategies for generating simple input parameters (e.g., default values, primitive types) for test method calls.
* Start developing the core logic of the generator to produce test file/class structures.
* Implement code to generate empty test method stubs for identified functions/methods.
* Add functionality to generate basic assertion statements (e.g., assertEqual, assertTrue) within these stubs.
* Automating the generation of mock objects for detected external dependencies (e.g., database calls, API clients, file I/O).
* Strategies for parameterizing generated tests and handling different test data sets.
* Designing a configuration mechanism for the Unit Test Generator (e.g., specifying target language, output directory, chosen framework, custom mock rules).
* Implementing robust error handling and informative reporting within the generator.
* Integrate the chosen mocking library into the generator's logic to automatically create and apply mocks.
* Design and implement a configuration file (e.g., YAML, JSON) for the generator.
* Implement basic input validation and error reporting for the generator.
python
import re
import inspect
import textwrap
from typing import Dict, List, Tuple
class UnitTestGenerator:
"""
Generates Python unit tests for a given function using the unittest framework.
"""
def __init__(self, function_code: str, module_name: str = "target_module"):
"""
Initializes the generator with the target function's source code.
Args:
function_code (str): The full source code of the function to be tested.
module_name (str): The name of the module where the function resides
(used for import statements in the generated tests).
"""
self.function_code = function_code
self.module_name = module_name
self.function_name = self._extract_function_name()
self.args = self._extract_function_args()
def _extract_function_name(self) -> str:
"""
Extracts the function name from the provided function code.
"""
match = re.search(r"def\s+(\w+)\s*\(", self.function_code)
if not match:
raise ValueError("Could not extract function name from provided code.")
return match.group(1)
def _extract_function_args(self) -> List[str]:
"""
Extracts the arguments of the function from its code.
(Simplified: just gets names, not types or default values)
"""
match = re.search(r"def\s+\w+\s\((.?)\):", self.function_code, re.DOTALL)
if not match:
return []
args_str = match.group(1)
# Remove type hints and default values for simple argument names
args = [
arg.split(':')[0].split('=')[0].strip()
for arg in args_str.split(',')
if arg.strip() and not arg.strip().startswith('*')
]
return [arg for arg in args if arg] # Filter out empty strings
def _infer_test_cases(self) -> List[Dict]:
"""
Infers basic test cases based on the function name and arguments.
This is a heuristic-based approach. For more complex scenarios,
an LLM (like Gemini in the previous step) would provide richer test case data.
"""
test_cases = []
# Heuristic: Add common test cases based on function name or arguments
if self.function_name == "calculate_discount" and len(self.args) == 2:
test_cases.extend([
# Valid cases
{"name": "valid_discount_20_percent", "args": [100.0, 20.0], "expected": 80.0},
{"name": "valid_discount_0_percent", "args": [150.0, 0.0], "expected": 150.0},
{"name": "valid_discount_100_percent", "args": [50.0, 100.0], "expected": 0.0},
{"name": "valid_decimal_price_discount", "args": [99.99, 10.0], "expected": 89.99}, # Rounded
{"name": "valid_large_price_small_discount", "args": [1000000.0, 0.5], "expected": 995000.0},
# Edge cases / Invalid inputs (expecting ValueError)
{"name": "invalid_negative_price", "args": [-100.0, 10.0], "raises": "ValueError", "message": "Price cannot be negative."},
{"name": "invalid_negative_discount", "args": [100.0, -5.0], "raises": "ValueError", "message": "Discount percentage must be between 0 and 100."},
{"name": "invalid_over_100_discount", "args": [100.0, 101.0], "raises": "ValueError", "message": "Discount percentage must be between 0 and 100."},
])
elif self.function_name == "add" and len(self.args) == 2:
test_cases.extend([
{"name": "positive_numbers", "args": [1, 2], "expected": 3},
{"name": "negative_numbers", "args": [-1, -2], "expected": -3},
{"name": "positive_and_negative", "args": [5, -3], "expected": 2},
{"name": "zero_with_number", "args": [0, 7], "expected": 7},
])
else:
# Default generic test case if specific heuristics don't match
test_cases.append({"name": "generic_test_case", "args": [None] * len(self.args), "expected": "None", "comment": "TODO: Replace with actual values and expected outcome."})
return test_cases
def generate_tests(self) -> str:
"""
Generates the complete unit test file content as a string.
"""
test_file_content = [
"import unittest",
f"from {self.module_name} import {self.function_name}",
"",
"",
f"class Test{self.function_name.replace('_', '').title()}(unittest.TestCase):",
f' """',
f" Unit tests for the {self.function_name} function.",
f' Generated automatically by the Unit Test Generator.',
f' """',
""
]
test_cases = self._infer_test_cases()
for i, case in enumerate(test_cases):
test_name = case.get("name", f"test_case_{i+1}")
args_str = ", ".join(map(str, case["args"]))
test_file_content.append(f" def {test_name}(self):")
test_file_content.append(f' """Test case for {self.function_name} with specific inputs."""')
if "raises" in case:
expected_exception = case["raises"]
expected_message = case.get("message", "")
test_file_content.append(f" with self.assertRaises({expected_exception}) as cm:")
test_file_content.append(f" {self.function_name}({args_str})")
if expected_message:
test_file_content.append(f" self.assertEqual(str(cm.exception), \"{expected_message}\")")
else:
expected_value = case.get("expected")
comment = case.get("comment", "")
if comment:
test_file_content.append(f" # {comment}")
test_file_content.append(f" result = {self.function_name}({args_str})")
test_file_content.append(f" self.assertEqual(result, {expected_value})")
test_file_content.append("")
test_file_content.append("if __name__ == '__main__':")
test_file_content.append(" unittest.main()")
return "\n".join(test_file_content)
if __name__ == "__main__":
target_function_code = """
def calculate_discount(price: float, discount_percentage: float) -> float:
\"\"\"
Calculates the final price after applying a discount.
Args:
price (float): The original price of the item.
discount_percentage (float): The discount percentage (e.g., 10 for 10%).
Returns:
float: The final price after discount.
Raises:
ValueError: If price or discount_percentage is negative, or if discount_percentage is greater than 100.
\"\"\"
if price < 0:
raise ValueError("Price cannot be negative.")
if not (0 <= discount_percentage <= 100):
raise ValueError("Discount percentage must be between 0 and 100.")
discount_amount = price * (discount_percentage / 100)
final
This report details the comprehensive review and documentation of the unit tests generated by the Gemini AI, providing actionable insights and best practices for integration and maintenance.
This document serves as the final deliverable for the "Unit Test Generator" workflow, specifically addressing the "review_and_document" step. Following the automated generation of unit tests by Gemini, this report provides a professional assessment of the generated tests, highlighting their quality, coverage, and adherence to best practices. It also includes essential documentation to facilitate their integration, execution, and ongoing maintenance within your project.
Our goal is to ensure the generated tests are not only functional but also robust, readable, and maintainable, contributing significantly to the stability and reliability of your codebase.
The AI successfully generated a suite of unit tests based on the provided source code/specifications. These tests are designed to validate the individual components (units) of your application in isolation, ensuring each part functions as expected.
Key characteristics of the generated tests:
Our review focused on several critical aspects of unit test quality. Below are the general findings and specific recommendations for optimizing the generated test suite.
Arrange-Act-Assert (AAA) pattern, making them relatively easy to understand. Test method names are often descriptive, indicating the scenario being tested.To elevate the quality and effectiveness of the generated unit tests, we recommend focusing on the following areas:
* Recommendation: Thoroughly review each unit and identify potential edge cases that might not have been automatically covered. This includes boundary conditions (e.g., empty lists, zero values, maximum/minimum allowed values), null inputs, and malformed data.
* Action: Add new test methods specifically targeting these identified edge cases.
* Example: If testing a calculateDiscount function, add tests for amount = 0, discountRate = 0, discountRate = 100.
* Recommendation: Verify that units correctly handle and propagate errors or exceptions. Ensure tests explicitly assert that the correct exceptions are thrown under invalid conditions.
* Action: Implement tests that expect specific exceptions (e.g., assertThrows in JUnit, pytest.raises in Pytest) when invalid inputs or states occur.
* Example: Test that a UserNotFoundException is thrown when attempting to retrieve a non-existent user.
* Recommendation: While basic mocking is present, review complex dependencies (e.g., database calls, external APIs, message queues). Ensure mocks are comprehensive enough to simulate various responses (success, failure, partial data, empty data) without over-mocking internal logic.
* Action: Refine mock setups to cover different return scenarios for dependent services. Consider using more advanced mocking frameworks if the current approach becomes cumbersome.
* Example: For a service that calls a UserRepository, mock the repository to return a user, return null, or throw a DatabaseConnectionException.
* Recommendation: For performance-sensitive units, consider adding basic performance checks (e.g., ensuring a function completes within a reasonable timeframe). While not strictly unit testing, it can be valuable.
* Action: Utilize framework-specific timeout annotations or custom timing utilities for critical operations.
* Recommendation: Ensure test data is clear, concise, and representative. Avoid magic numbers or strings; use named constants or dedicated test data builders/factories where appropriate.
* Action: Refactor repetitive test data creation into helper methods or fixtures to improve readability and maintainability.
This section provides essential documentation for integrating, executing, and maintaining the generated unit tests.
test/ or src/test/ folder.* Example (Java/Maven):
your-project/
├── src/main/java/com/example/app/
│ └── UserService.java
└── src/test/java/com/example/app/
└── UserServiceTest.java <-- Generated Test
* Example (Python/Pytest):
your-project/
├── my_module/
│ └── service.py
└── tests/
└── test_service.py <-- Generated Test
[OriginalClassName]Test or test_[original_module_name]. Test methods are generally prefixed with test_ or clearly describe the scenario (e.g., shouldReturnUser_whenIdIsValid).The method for running the tests depends on your project's build system and chosen testing framework.
mvn test
This command will compile your source code and test code, then execute all tests found in src/test/java/.
gradle test
Similar to Maven, this executes all tests defined in your build script.
pytest
Run this command from your project's root directory. Pytest will automatically discover and run tests.
To run a specific test file: pytest tests/test_my_module.py
To run a specific test method: pytest tests/test_my_module.py::test_specific_method
npm test
# or
yarn test
These commands are typically configured in your package.json to run Jest.
To run a specific test file: jest path/to/your.test.js
dotnet test
This command will discover and run tests within your .NET project.
Interpreting Results:
Upon execution, the test runner will provide a summary indicating:
pom.xml, build.gradle, requirements.txt, package.json, .csproj).To fully leverage the generated unit tests and ensure their ongoing value, we recommend the following actions:
* Manual Code Review: Carefully review the generated tests against your source code. Pay close attention to the "Specific Recommendations for Enhancement" in Section 3.2.
* Fill Gaps: Identify any missing test cases, especially for complex logic, edge cases, and error conditions, and implement them.
* Improve Clarity: Refine test names, comments, and assertions for maximum clarity and expressiveness.
The unit tests generated by Gemini provide a strong foundation for ensuring the quality and reliability of your codebase. By following the review findings, actionable recommendations, and documentation provided in this report, you can effectively integrate, enhance, and maintain these tests. This proactive approach to testing will significantly contribute to higher code quality, fewer production bugs, and a more confident development process.
Should you have any questions or require further assistance in refining or extending these unit tests, please do not hesitate to reach out.