Test Fixture Migration: Documentation Review and Status Assessment

Session Date: 2025-11-25 Project: ast-grep-mcp Focus: Review test fixture migration progress and tool registration status

Executive Summary

Reviewed comprehensive documentation on the ongoing test fixture migration effort for the ast-grep-mcp project. The project has successfully completed Phase 2 of fixture migration, converting test_apply_deduplication.py from setup_method patterns to modern pytest fixtures, achieving an 18.4% code reduction while maintaining 100% test pass rate. However, a critical architectural limitation has been identified: only 11 of 27 MCP tools (41%) can be registered in the test fixture system due to nested function definitions using decorators.

Key Findings:

• ✅ Phase 2 migration completed: 24/24 tests passing
• ✅ 18.4% code reduction (130 lines removed)
• ❌ 16 of 27 tools blocked from registration (59%)
• 📋 Clear path forward documented with 3 solution options

Documentation Reviewed

1. TOOL-REGISTRATION-STATUS.md

This document provides a comprehensive analysis of MCP tool registration in the test fixture system.

Current Registration Status:

Category	Registered	Total	Percentage
Code Search	4	4	100% ✅
Code Rewrite	3	3	100% ✅
Deduplication	4	4	100% ✅
Schema.org	0	8	0% ❌
Complexity	0	2	0% ❌
Code Quality	0	3	0% ❌
Testing	0	3	0% ❌
Total	11	27	41%

Why 11 Tools Work:

• Implemented as standalone *_impl or *_tool functions
• Directly importable from service modules
• Examples:
  • ast_grep_mcp.features.search.service.find_code_impl
  • ast_grep_mcp.features.rewrite.backup.restore_backup
  • ast_grep_mcp.features.deduplication.tools.find_duplication_tool

Why 16 Tools Don’t Work:

The blocked tools use this problematic pattern:

def register_complexity_tools(mcp: FastMCP) -> None:
    @mcp.tool()
    def analyze_complexity(...):
        # 100+ lines of logic defined inline
        ...

Problems:

1. Functions are nested inside register_*_tools()
2. Functions are not exported from the module
3. Functions only exist after calling register_*_tools(mcp)
4. Cannot be directly imported for test fixtures

Solution Options Documented:

Option 1: Extract Tool Logic (Recommended)

• Extract nested functions into standalone *_tool() functions
• Register them with mcp.tool()(function_name)
• Effort: 2-4 hours for all 16 tools
• Benefits: Clean, testable, importable, maintainable

Option 2: Use Tool Wrappers

• Create wrapper functions that replicate tool logic
• Effort: 1-2 hours for all 16 tools
• Drawbacks: Code duplication, maintenance burden

Option 3: Integration Testing Only

• Don’t register these tools for unit tests
• Use integration tests with actual MCP server
• Drawbacks: Slower tests, no unit test coverage

Test Impact:

• ✅ Working: test_apply_deduplication.py (22/24 passing), test_rewrite.py (33/33 passing)
• ❌ Blocked: Schema.org, Complexity, Quality, and Testing tool tests
• Estimated: 30-40% of test suite blocked

2. fixture-migration-phase2-report.md

Detailed report on successful completion of Phase 2 fixture migration.

Migration Results:

Metric	Value
Lines of code	708 → 578 (-130 lines, -18.4%)
Test classes migrated	4
Test methods	24 (all preserved)
Tests passing	24/24 (100%)
Execution time	0.63s

Classes Migrated:

1. TestOrchestrationHelperFunctions (Phase 1)
   • Complexity: Easiest
   • Tests: 6 methods
   • Changes: Removed empty setup_method, added project_folder fixture
2. TestApplyDeduplication (Phase 2)
   • Complexity: Medium
   • Tests: 6 methods
   • Fixtures: project_folder, simple_test_files, apply_deduplication_tool, refactoring_plan_factory
   • Helper removed: _create_refactoring_plan() replaced with factory fixture
3. TestBackupIntegration (Phase 3)
   • Complexity: Medium
   • Tests: 7 methods
   • Fixtures: project_folder, backup_test_files, apply_deduplication_tool, refactoring_plan_factory
   • Helper removed: _create_plan_with_content() replaced with factory fixture
4. TestPhase33MultiFileOrchestration (Phase 4)
   • Complexity: High (subdirectories, complex content)
   • Tests: 5 methods
   • Fixtures: project_folder, orchestration_test_files, apply_deduplication_tool, refactoring_plan_factory
   • Helper removed: _create_plan_with_extraction() replaced with factory fixture

Migration Pattern:

Before (setup_method pattern):

class TestClass:
    def setup_method(self) -> None:
        self.temp_dir = tempfile.mkdtemp()
        self.project_folder = self.temp_dir
        # Create files manually
        self.apply_deduplication = main.mcp.tools.get("apply_deduplication")

    def teardown_method(self) -> None:
        shutil.rmtree(self.temp_dir, ignore_errors=True)

    def _create_refactoring_plan(self, ...):
        # Helper method duplicated across classes

After (fixture pattern):

class TestClass:
    def test_something(
        self,
        project_folder,
        simple_test_files,
        apply_deduplication_tool,
        refactoring_plan_factory
    ):
        # Use fixtures directly
        plan = refactoring_plan_factory(files=[simple_test_files["file1"]])
        result = apply_deduplication_tool(
            project_folder=str(project_folder),
            refactoring_plan=plan
        )

Fixtures Used from conftest.py:

• project_folder - Temporary directory with automatic cleanup
• simple_test_files - Basic test file setup
• backup_test_files - Files with original content tracking
• orchestration_test_files - Complex subdirectory structure
• apply_deduplication_tool - MCP tool access (module-scoped)
• refactoring_plan_factory - Factory for creating refactoring plans

Benefits Achieved:

1. Cleaner code: No manual setup/teardown, pytest handles cleanup automatically
2. Better isolation: Each test gets fresh fixtures
3. Reduced duplication: Helper methods replaced with shared factory fixture
4. More readable: Test dependencies explicit in function signatures
5. Less code: 130 lines removed (18.4% reduction)
6. Better performance: Module-scoped tool fixtures reduce import overhead

Import Cleanup:

• Removed shutil - No longer needed (pytest handles cleanup)
• Removed tempfile - No longer needed (tmp_path fixture used)

Key Insights

1. Successful Migration Strategy

The Phase 2 migration demonstrates a well-executed strategy:

• Started with simplest class (TestOrchestrationHelperFunctions)
• Progressively tackled more complex classes
• Used factory fixtures to eliminate duplicate helper methods
• Maintained 100% test pass rate throughout

2. Architectural Limitation Identified

The documentation clearly identifies a critical architectural issue:

• 59% of tools cannot be registered due to decorator pattern
• This blocks 30-40% of the test suite
• Root cause: nested function definitions inside registration functions

3. Clear Path Forward

Three solution options documented with effort estimates:

• Option 1 (recommended): Refactor 16 tools to extract logic (2-4 hours)
• Option 2: Create wrappers (1-2 hours, but creates tech debt)
• Option 3: Skip unit tests, use integration tests only

4. Documentation Quality

Both documents demonstrate excellent technical documentation:

• Clear problem statements
• Concrete metrics and results
• Code examples showing before/after
• Actionable recommendations
• Realistic effort estimates

Lessons Learned

From fixture-migration-phase2-report.md:

1. Start with simplest class first - Build confidence and establish patterns
2. Factory fixtures are powerful - One flexible factory replaced 3 helper methods
3. Module-scoped fixtures save time - Tool registration happens once, not per test
4. Explicit is better - Fixture parameters make dependencies clear

From TOOL-REGISTRATION-STATUS.md:

1. Decorator patterns can block testability - Nested functions aren’t importable
2. Document limitations early - Prevents wasted effort
3. Provide multiple solution options - Allows informed decision-making
4. Quantify impact - “30-40% of test suite blocked” is actionable

Recommended Next Steps

Based on the documentation review:

Immediate (Phase 3)

• ✅ Continue fixture migration with 11 working tools
• ✅ Document limitation in test files
• ✅ Focus on test files using search/rewrite/deduplication tools

Next Sprint (Tool Refactoring)

• Create GitHub issue for Option 1 refactoring
• Refactor 16 blocked tools to extract logic into standalone functions
• Update tool registration to use extracted functions
• Validate all 27 tools can be registered
• Resume fixture migration for blocked test files

Pattern to Apply

For each of the 16 blocked tools:

Before:

# src/ast_grep_mcp/features/complexity/tools.py
def register_complexity_tools(mcp: FastMCP) -> None:
    @mcp.tool()
    def analyze_complexity(...):
        # 100+ lines of logic

After:

# src/ast_grep_mcp/features/complexity/tools.py

def analyze_complexity_tool(...):
    """Standalone function with tool logic."""
    # 100+ lines of logic
    return result

def register_complexity_tools(mcp: FastMCP) -> None:
    """Register complexity tools with MCP server."""
    mcp.tool()(analyze_complexity_tool)

Benefits:

• analyze_complexity_tool can be imported for tests
• Logic remains unchanged
• Registration still works
• Test fixtures can access all tools

Conclusion

The ast-grep-mcp project has made excellent progress on test fixture migration, with Phase 2 successfully completed. The documentation reveals both the success of the migration (18.4% code reduction, 100% test pass rate) and a critical architectural limitation blocking 59% of tools from registration.

The path forward is clear: implement Option 1 (extract tool logic) for the 16 blocked tools, then resume fixture migration for the remaining test files. This two-phase approach allows continued progress while properly addressing the architectural issue.

The quality of documentation demonstrates mature engineering practices: clear problem identification, quantified impact analysis, multiple solution options with trade-offs, and realistic effort estimates.

References

• docs/TOOL-REGISTRATION-STATUS.md - Tool registration analysis
• tests/docs/fixture-migration-phase2-report.md - Phase 2 migration results
• tests/unit/test_apply_deduplication.py - Migrated test file
• tests/unit/conftest.py - Shared test fixtures
• src/ast_grep_mcp/features/*/tools.py - Tool registration modules

Tags: ast-grep-mcp, fixtures, migration-tracking, pytest, test-architecture

Categories: code-quality, documentation, testing

Updated: November 25, 2025