git-pandas(1)
| GIT-PANDAS(1) | git-pandas | GIT-PANDAS(1) |
NAME
git-pandas - git-pandas Documentation
Git-Pandas is a Python library that transforms Git repository data into pandas DataFrames, making it easy to analyze your codebase's history and development patterns. Built on top of GitPython, it provides a simple interface for extracting insights from your Git repositories. [image: Cumulative Blame Visualization] [image]
QUICK START
Install Git-Pandas using pip:
pip install git-pandas
Basic Usage
Analyze a single repository:
from gitpandas import Repository
# Create a repository instance
repo = Repository('/path/to/repo')
# Get commit history with filtering
commits_df = repo.commit_history(
branch='main',
ignore_globs=['*.pyc'],
include_globs=['*.py']
)
# Analyze blame information
blame_df = repo.blame(by='repository')
# Calculate bus factor
bus_factor_df = repo.bus_factor()
Analyze multiple repositories:
from gitpandas import ProjectDirectory
project = ProjectDirectory('/path/to/project')
KEY FEATURES
- Repository Analysis: Extract commit history, file changes, and blame information
- Project Insights: Calculate bus factor and analyze repository metrics
- Multi-Repository Support: Analyze multiple repositories together
- Remote Operations: Safely fetch changes from remote repositories
- Cache Warming: Pre-populate caches for improved performance
- Bulk Operations: Efficiently process multiple repositories in parallel
- Performance Optimization: Advanced caching support and glob-based filtering
CORE COMPONENTS
The library is built around two main components:
Repository
A wrapper around a single Git repository that provides:
- Commit history analysis with filtering options
- File change tracking and blame information
- Branch existence checking and repository status
- Bus factor calculation and repository metrics
- Punchcard statistics generation
ProjectDirectory
A collection of Git repositories that enables:
- Analysis across multiple repositories
- Aggregated metrics and statistics
- Project-level insights
COMMON PARAMETERS
Most analysis methods support these filtering parameters:
- branch: Branch to analyze (defaults to repository's default branch)
- limit: Maximum number of commits to analyze
- days: Limit analysis to last N days
- ignore_globs: List of glob patterns for files to ignore
- include_globs: List of glob patterns for files to include
- by: How to group results (usually 'repository' or 'file')
DOCUMENTATION
For detailed information about the components and their usage, see:
Repository
The Repository class provides an interface for analyzing a single Git repository. It can be created from either a local or remote repository.
Overview
The Repository class offers methods for:
- Commit history analysis with filtering options
- File change tracking and blame information
- Branch existence checking and repository status
- Bus factor calculation and repository metrics
- Punchcard statistics generation
Creating a Repository
You can create a Repository object in two ways:
Local Repository
Create a Repository from a local Git repository:
from gitpandas import Repository repo = Repository(
working_dir='/path/to/repo/',
verbose=True,
default_branch='main' # Optional, will auto-detect if not specified )
The directory must contain a .git directory. Subdirectories are not searched.
Remote Repository
Create a Repository from a remote Git repository:
from gitpandas import Repository repo = Repository(
working_dir='git://github.com/user/repo.git',
verbose=True,
default_branch='main' # Optional, will auto-detect if not specified )
The repository will be cloned locally into a temporary directory. This can be slow for large repositories.
Available Methods
Core Analysis
# Commit history analysis repo.commit_history(
branch=None, # Branch to analyze
limit=None, # Maximum number of commits
days=None, # Limit to last N days
ignore_globs=None, # Files to ignore
include_globs=None # Files to include ) # File change history repo.file_change_history(
branch=None,
limit=None,
days=None,
ignore_globs=None,
include_globs=None ) # Blame analysis repo.blame(
rev="HEAD", # Revision to analyze
committer=True, # Group by committer (False for author)
by="repository", # Group by 'repository' or 'file'
ignore_globs=None,
include_globs=None ) # Bus factor analysis repo.bus_factor(
by="repository", # How to group results ('repository' or 'file')
ignore_globs=None,
include_globs=None ) # Commit pattern analysis repo.punchcard(
branch=None,
limit=None,
days=None,
by=None, # Additional grouping
normalize=None, # Normalize values
ignore_globs=None,
include_globs=None )
Repository Information
# List files in repository repo.list_files(rev="HEAD") # Check branch existence repo.has_branch(branch) # Check if repository is bare repo.is_bare() # Check for coverage information repo.has_coverage() repo.coverage() # Get specific commit content repo.get_commit_content(
rev, # Revision to analyze
ignore_globs=None,
include_globs=None )
Common Parameters
Most analysis methods support these filtering parameters:
- branch: Branch to analyze (defaults to repository's default branch)
- limit: Maximum number of commits to analyze
- days: Limit analysis to last N days
- ignore_globs: List of glob patterns for files to ignore
- include_globs: List of glob patterns for files to include
- by: How to group results (usually 'repository' or 'file')
API Reference
- class gitpandas.repository.Repository(working_dir=None, verbose=False, tmp_dir=None, cache_backend=None, labels_to_add=None, default_branch=None)
- Bases: object
A class for analyzing a single git repository.
This class provides functionality to analyze a git repository, whether it is a local repository or a remote repository that needs to be cloned. It offers methods for analyzing commit history, blame information, file changes, and other git metrics.
- Args:
- working_dir (Optional[str]): Path to the git repository:
- If None: Uses current working directory
- If local path: Path must contain a .git directory
- If git URL: Repository will be cloned to a temporary directory
verbose (bool, optional): Whether to print verbose output. Defaults to False. tmp_dir (Optional[str]): Directory to clone remote repositories into. Created if not provided. cache_backend (Optional[object]): Cache backend instance from gitpandas.cache labels_to_add (Optional[List[str]]): Extra labels to add to output DataFrames default_branch (Optional[str]): Name of the default branch to use. If None, will try to detect
- Attributes:
- verbose (bool): Whether verbose output is enabled git_dir (str): Path to the git repository repo (git.Repo): GitPython Repo instance cache_backend (Optional[object]): Cache backend being used _labels_to_add (List[str]): Labels to add to DataFrames _git_repo_name (Optional[str]): Repository name for remote repos default_branch (str): Name of the default branch
- Raises:
- ValueError: If default_branch is None and neither 'main' nor 'master' branch exists
- Examples:
-
>>> # Create from local repository >>> repo = Repository('/path/to/repo')>>> # Create from remote repository >>> repo = Repository('git://github.com/user/repo.git') - Note:
- When using remote repositories, they will be cloned to temporary directories. This can be slow for large repositories.
- __init__(working_dir=None, verbose=False, tmp_dir=None, cache_backend=None, labels_to_add=None, default_branch=None)
- Initialize a Repository instance.
- Args:
- working_dir (Optional[str]): Path to the git repository:
- If None: Uses current working directory
- If local path: Path must contain a .git directory
- If git URL: Repository will be cloned to a temporary directory
verbose (bool, optional): Whether to print verbose output. Defaults to False. tmp_dir (Optional[str]): Directory to clone remote repositories into. Created if not provided. cache_backend (Optional[object]): Cache backend instance from gitpandas.cache labels_to_add (Optional[List[str]]): Extra labels to add to output DataFrames default_branch (Optional[str]): Name of the default branch to use. If None, will try to detect
- Raises:
- ValueError: If default_branch is None and neither 'main' nor 'master' branch exists
- is_bare(*args, **kwargs)
- has_coverage(*args, **kwargs)
- coverage(*args, **kwargs)
- hours_estimate(*args, **kwargs)
- commit_history(*args, **kwargs)
- file_change_history(*args, **kwargs)
- file_change_rates(*args, **kwargs)
- blame(*args, **kwargs)
- revs(*args, **kwargs)
- cumulative_blame(*args, **kwargs)
- parallel_cumulative_blame(*args, **kwargs)
- branches(*args, **kwargs)
- get_branches_by_commit(*args, **kwargs)
- property repo_name
- __str__()
- Returns a human-readable string representation of the repository.
- Returns:
- str: String in format 'git repository: {name} at: {path}'
- get_commit_content(*args, **kwargs)
- get_file_content(*args, **kwargs)
- list_files(*args, **kwargs)
- __repr__()
- Returns a unique string representation of the repository.
- Returns:
- str: The absolute path to the repository
- bus_factor(*args, **kwargs)
- file_owner(*args, **kwargs)
- punchcard(*args, **kwargs)
- has_branch(*args, **kwargs)
- file_detail(*args, **kwargs)
- time_between_revs(rev1, rev2)
- Calculates the time difference in days between two revisions.
- Args:
- rev1 (str): The first revision (commit hash or tag). rev2 (str): The second revision (commit hash or tag).
- Returns:
- float: The absolute time difference in days between the two revisions.
- Note:
- The result is always non-negative (absolute value).
- diff_stats_between_revs(rev1, rev2, ignore_globs=None, include_globs=None)
- Computes diff statistics between two revisions.
Calculates the total insertions, deletions, net line change, and number of files changed between two arbitrary revisions (commits or tags). Optionally filters files using glob patterns.
- Args:
- rev1 (str): The base revision (commit hash or tag). rev2 (str): The target revision (commit hash or tag). ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore. include_globs (Optional[List[str]]): List of glob patterns for files to include.
- Returns:
- dict: A dictionary with keys:
- 'insertions' (int): Total lines inserted.
- 'deletions' (int): Total lines deleted.
- 'net' (int): Net lines changed (insertions - deletions).
- 'files_changed' (int): Number of files changed.
- 'files' (List[str]): List of changed file paths.
- Note:
- Binary files or files that cannot be parsed are skipped. If both ignore_globs and include_globs are provided, files must match an include pattern and not match any ignore patterns to be included.
- committers_between_revs(rev1, rev2, ignore_globs=None, include_globs=None)
- Finds unique committers and authors between two revisions.
Iterates through all commits between two revisions (exclusive of rev1, inclusive of rev2) and returns the unique committers and authors who contributed, filtered by file globs if provided.
- Args:
- rev1 (str): The base revision (commit hash or tag). rev2 (str): The target revision (commit hash or tag). ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore. include_globs (Optional[List[str]]): List of glob patterns for files to include.
- Returns:
- dict: A dictionary with keys:
- 'committers' (List[str]): Sorted list of unique committer names.
- 'authors' (List[str]): Sorted list of unique author names.
- Note:
- Only commits that touch files matching the glob filters are considered. The range is interpreted as Git does: rev1..rev2 means commits reachable from rev2 but not rev1.
- files_changed_between_revs(rev1, rev2, ignore_globs=None, include_globs=None)
- Lists files changed between two revisions.
Returns a sorted list of all files changed between two arbitrary revisions (commits or tags), optionally filtered by glob patterns.
- Args:
- rev1 (str): The base revision (commit hash or tag). rev2 (str): The target revision (commit hash or tag). ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore. include_globs (Optional[List[str]]): List of glob patterns for files to include.
- Returns:
- List[str]: Sorted list of file paths changed between the two revisions.
- Note:
- If both ignore_globs and include_globs are provided, files must match an include pattern and not match any ignore patterns to be included.
- release_tag_summary(*args, **kwargs)
- safe_fetch_remote(remote_name='origin', prune=False, dry_run=False)
- Safely fetch changes from remote repository.
Fetches the latest changes from a remote repository without modifying the working directory. This is a read-only operation that only updates remote-tracking branches.
- Args:
- remote_name (str, optional): Name of remote to fetch from. Defaults to 'origin'. prune (bool, optional): Remove remote-tracking branches that no longer exist on remote.
- dry_run (bool, optional): Show what would be fetched without actually fetching.
- Defaults to False.
- Returns:
- dict: Fetch results with keys:
- success (bool): Whether the fetch was successful
- message (str): Status message or error description
- remote_exists (bool): Whether the specified remote exists
- changes_available (bool): Whether new changes were fetched
- error (Optional[str]): Error message if fetch failed
- Note:
- This method is safe as it only fetches remote changes and never modifies the working directory or current branch. It will not perform any merges, rebases, or checkouts.
- warm_cache(methods=None, **kwargs)
- Pre-populate cache with commonly used data.
Executes a set of commonly used repository analysis methods to populate the cache, improving performance for subsequent calls. Only methods that support caching will be executed.
- Args:
- methods (Optional[List[str]]): List of method names to pre-warm. If None,
- uses a default set of commonly used methods. Available methods: - 'commit_history': Load commit history - 'branches': Load branch information - 'tags': Load tag information - 'blame': Load blame information - 'file_detail': Load file details - 'list_files': Load file listing - 'file_change_rates': Load file change statistics
- **
- kwargs: Additional keyword arguments to pass to the methods. Common arguments include: - branch: Branch to analyze (default: repository's default branch) - limit: Limit number of commits to analyze - ignore_globs: List of glob patterns to ignore - include_globs: List of glob patterns to include
- Returns:
- dict: Results of cache warming operations with keys:
- success (bool): Whether cache warming was successful
- methods_executed (List[str]): List of methods that were executed
- methods_failed (List[str]): List of methods that failed
- cache_entries_created (int): Number of cache entries created
- execution_time (float): Total execution time in seconds
- errors (List[str]): List of error messages for failed methods
- Note:
- This method will only execute methods if a cache backend is configured. If no cache backend is available, it will return immediately with a success status but no methods executed.
- invalidate_cache(keys=None, pattern=None)
- Invalidate specific cache entries or all cache entries for this repository.
- Args:
- keys (Optional[List[str]]): List of specific cache keys to invalidate pattern (Optional[str]): Pattern to match cache keys (supports * wildcard)
- Returns:
- int: Number of cache entries invalidated
- Note:
- If both keys and pattern are None, all cache entries for this repository are invalidated. Cache keys are automatically prefixed with repository name.
- get_cache_stats()
- Get cache statistics for this repository.
- Returns:
- dict: Cache statistics including repository-specific and global cache information
- class gitpandas.repository.GitFlowRepository
- Bases: Repository
A special case where git flow is followed, so we know something about the branching scheme
- blame(*args, **kwargs)
- branches(*args, **kwargs)
- bus_factor(*args, **kwargs)
- commit_history(*args, **kwargs)
- committers_between_revs(rev1, rev2, ignore_globs=None, include_globs=None)
- Finds unique committers and authors between two revisions.
Iterates through all commits between two revisions (exclusive of rev1, inclusive of rev2) and returns the unique committers and authors who contributed, filtered by file globs if provided.
- Args:
- rev1 (str): The base revision (commit hash or tag). rev2 (str): The target revision (commit hash or tag). ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore. include_globs (Optional[List[str]]): List of glob patterns for files to include.
- Returns:
- dict: A dictionary with keys:
- 'committers' (List[str]): Sorted list of unique committer names.
- 'authors' (List[str]): Sorted list of unique author names.
- Note:
- Only commits that touch files matching the glob filters are considered. The range is interpreted as Git does: rev1..rev2 means commits reachable from rev2 but not rev1.
- coverage(*args, **kwargs)
- cumulative_blame(*args, **kwargs)
- diff_stats_between_revs(rev1, rev2, ignore_globs=None, include_globs=None)
- Computes diff statistics between two revisions.
Calculates the total insertions, deletions, net line change, and number of files changed between two arbitrary revisions (commits or tags). Optionally filters files using glob patterns.
- Args:
- rev1 (str): The base revision (commit hash or tag). rev2 (str): The target revision (commit hash or tag). ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore. include_globs (Optional[List[str]]): List of glob patterns for files to include.
- Returns:
- dict: A dictionary with keys:
- 'insertions' (int): Total lines inserted.
- 'deletions' (int): Total lines deleted.
- 'net' (int): Net lines changed (insertions - deletions).
- 'files_changed' (int): Number of files changed.
- 'files' (List[str]): List of changed file paths.
- Note:
- Binary files or files that cannot be parsed are skipped. If both ignore_globs and include_globs are provided, files must match an include pattern and not match any ignore patterns to be included.
- file_change_history(*args, **kwargs)
- file_change_rates(*args, **kwargs)
- file_detail(*args, **kwargs)
- file_owner(*args, **kwargs)
- files_changed_between_revs(rev1, rev2, ignore_globs=None, include_globs=None)
- Lists files changed between two revisions.
Returns a sorted list of all files changed between two arbitrary revisions (commits or tags), optionally filtered by glob patterns.
- Args:
- rev1 (str): The base revision (commit hash or tag). rev2 (str): The target revision (commit hash or tag). ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore. include_globs (Optional[List[str]]): List of glob patterns for files to include.
- Returns:
- List[str]: Sorted list of file paths changed between the two revisions.
- Note:
- If both ignore_globs and include_globs are provided, files must match an include pattern and not match any ignore patterns to be included.
- get_branches_by_commit(*args, **kwargs)
- get_cache_stats()
- Get cache statistics for this repository.
- Returns:
- dict: Cache statistics including repository-specific and global cache information
- get_commit_content(*args, **kwargs)
- get_file_content(*args, **kwargs)
- has_branch(*args, **kwargs)
- has_coverage(*args, **kwargs)
- hours_estimate(*args, **kwargs)
- invalidate_cache(keys=None, pattern=None)
- Invalidate specific cache entries or all cache entries for this repository.
- Args:
- keys (Optional[List[str]]): List of specific cache keys to invalidate pattern (Optional[str]): Pattern to match cache keys (supports * wildcard)
- Returns:
- int: Number of cache entries invalidated
- Note:
- If both keys and pattern are None, all cache entries for this repository are invalidated. Cache keys are automatically prefixed with repository name.
- is_bare(*args, **kwargs)
- list_files(*args, **kwargs)
- parallel_cumulative_blame(*args, **kwargs)
- punchcard(*args, **kwargs)
- release_tag_summary(*args, **kwargs)
- property repo_name
- revs(*args, **kwargs)
- safe_fetch_remote(remote_name='origin', prune=False, dry_run=False)
- Safely fetch changes from remote repository.
Fetches the latest changes from a remote repository without modifying the working directory. This is a read-only operation that only updates remote-tracking branches.
- Args:
- remote_name (str, optional): Name of remote to fetch from. Defaults to 'origin'. prune (bool, optional): Remove remote-tracking branches that no longer exist on remote.
- dry_run (bool, optional): Show what would be fetched without actually fetching.
- Defaults to False.
- Returns:
- dict: Fetch results with keys:
- success (bool): Whether the fetch was successful
- message (str): Status message or error description
- remote_exists (bool): Whether the specified remote exists
- changes_available (bool): Whether new changes were fetched
- error (Optional[str]): Error message if fetch failed
- Note:
- This method is safe as it only fetches remote changes and never modifies the working directory or current branch. It will not perform any merges, rebases, or checkouts.
- time_between_revs(rev1, rev2)
- Calculates the time difference in days between two revisions.
- Args:
- rev1 (str): The first revision (commit hash or tag). rev2 (str): The second revision (commit hash or tag).
- Returns:
- float: The absolute time difference in days between the two revisions.
- Note:
- The result is always non-negative (absolute value).
- warm_cache(methods=None, **kwargs)
- Pre-populate cache with commonly used data.
Executes a set of commonly used repository analysis methods to populate the cache, improving performance for subsequent calls. Only methods that support caching will be executed.
- Args:
- methods (Optional[List[str]]): List of method names to pre-warm. If None,
- uses a default set of commonly used methods. Available methods: - 'commit_history': Load commit history - 'branches': Load branch information - 'tags': Load tag information - 'blame': Load blame information - 'file_detail': Load file details - 'list_files': Load file listing - 'file_change_rates': Load file change statistics
- **
- kwargs: Additional keyword arguments to pass to the methods. Common arguments include: - branch: Branch to analyze (default: repository's default branch) - limit: Limit number of commits to analyze - ignore_globs: List of glob patterns to ignore - include_globs: List of glob patterns to include
- Returns:
- dict: Results of cache warming operations with keys:
- success (bool): Whether cache warming was successful
- methods_executed (List[str]): List of methods that were executed
- methods_failed (List[str]): List of methods that failed
- cache_entries_created (int): Number of cache entries created
- execution_time (float): Total execution time in seconds
- errors (List[str]): List of error messages for failed methods
- Note:
- This method will only execute methods if a cache backend is configured. If no cache backend is available, it will return immediately with a success status but no methods executed.
- __init__()
- Initialize a Repository instance.
- Args:
- working_dir (Optional[str]): Path to the git repository:
- If None: Uses current working directory
- If local path: Path must contain a .git directory
- If git URL: Repository will be cloned to a temporary directory
verbose (bool, optional): Whether to print verbose output. Defaults to False. tmp_dir (Optional[str]): Directory to clone remote repositories into. Created if not provided. cache_backend (Optional[object]): Cache backend instance from gitpandas.cache labels_to_add (Optional[List[str]]): Extra labels to add to output DataFrames default_branch (Optional[str]): Name of the default branch to use. If None, will try to detect
- Raises:
- ValueError: If default_branch is None and neither 'main' nor 'master' branch exists
Project Directory
The ProjectDirectory class enables analysis across multiple Git repositories. It can aggregate metrics and insights from multiple repositories into a single output.
Overview
The ProjectDirectory class provides:
- Analysis across multiple repositories
- Aggregated metrics and statistics
- Project-level insights
- Multi-repository bus factor analysis
- Consolidated commit history and blame information
Creating a ProjectDirectory
You can create a ProjectDirectory object in three ways:
Directory of Repositories
Create a ProjectDirectory from a directory containing multiple repositories:
from gitpandas import ProjectDirectory project = ProjectDirectory(
working_dir='/path/to/dir/',
ignore_repos=['repo_to_ignore'],
verbose=True,
default_branch='main' # Optional, will auto-detect if not specified )
The ignore_repos parameter can be a list of repository names to exclude. This method uses os.walk to search for .git directories recursively.
Explicit Local Repositories
Create a ProjectDirectory from a list of local repositories:
from gitpandas import ProjectDirectory project = ProjectDirectory(
working_dir=['/path/to/repo1/', '/path/to/repo2/'],
ignore_repos=['repo_to_ignore'],
verbose=True,
default_branch='main' # Optional, will auto-detect if not specified )
Explicit Remote Repositories
Create a ProjectDirectory from a list of remote repositories:
from gitpandas import ProjectDirectory project = ProjectDirectory(
working_dir=['git://github.com/user/repo1.git', 'git://github.com/user/repo2.git'],
ignore_repos=['repo_to_ignore'],
verbose=True,
default_branch='main' # Optional, will auto-detect if not specified )
Available Methods
Core Analysis
# Commit history across repositories project.commit_history(
branch=None, # Branch to analyze
limit=None, # Maximum number of commits
days=None, # Limit to last N days
ignore_globs=None, # Files to ignore
include_globs=None # Files to include ) # File change history across repositories project.file_change_history(
branch=None,
limit=None,
days=None,
ignore_globs=None,
include_globs=None ) # Blame analysis across repositories project.blame(
rev="HEAD", # Revision to analyze
committer=True, # Group by committer (False for author)
by="repository", # Group by 'repository' or 'file'
ignore_globs=None,
include_globs=None ) # Bus factor analysis across repositories project.bus_factor(
by="repository", # How to group results ('projectd', 'repository', or 'file')
ignore_globs=None,
include_globs=None )
Common Parameters
Most analysis methods support these filtering parameters:
- branch: Branch to analyze (defaults to repository's default branch)
- limit: Maximum number of commits to analyze
- days: Limit analysis to last N days
- ignore_globs: List of glob patterns for files to ignore
- include_globs: List of glob patterns for files to include
- by: How to group results (usually 'repository' or 'file')
API Reference
- class gitpandas.project.ProjectDirectory(working_dir=None, ignore_repos=None, verbose=True, tmp_dir=None, cache_backend=None, default_branch='main')
- Bases: object
A class for analyzing multiple git repositories in a directory or from explicit paths.
This class provides functionality to analyze multiple git repositories together, whether they are local repositories in a directory, explicitly specified local repositories, or remote repositories that need to be cloned. It offers methods for analyzing commit history, blame information, file changes, and other git metrics across all repositories.
- Args:
- working_dir (Union[str, List[str], None]): The source of repositories to analyze:
- If None: Uses current working directory to find repositories
- If str: Path to directory containing git repositories
- If List[str]: List of paths to git repositories or Repository instances
ignore_repos (Optional[List[str]]): List of repository names to ignore verbose (bool, optional): Whether to print verbose output. Defaults to True. tmp_dir (Optional[str]): Directory to clone remote repositories into. Created if not provided. cache_backend (Optional[object]): Cache backend instance from gitpandas.cache default_branch (str, optional): Name of the default branch to use. Defaults to 'main'.
- Attributes:
- repo_dirs (Union[set, list]): Set of repository directories or list of Repository instances repos (List[Repository]): List of Repository objects being analyzed
- Examples:
-
>>> # Create from directory containing repos >>> pd = ProjectDirectory(working_dir='/path/to/repos')>>> # Create from explicit local repos >>> pd = ProjectDirectory(working_dir=['/path/to/repo1', '/path/to/repo2'])>>> # Create from remote repos >>> pd = ProjectDirectory(working_dir=['git://github.com/user/repo.git']) - Note:
- When using remote repositories, they will be cloned to temporary directories. This can be slow for large repositories.
Methods
- __init__(working_dir=None, ignore_repos=None, verbose=True, tmp_dir=None, cache_backend=None, default_branch='main')
- Initialize a ProjectDirectory instance.
- Args:
- working_dir (Union[str, List[str], None]): The source of repositories to analyze:
- If None: Uses current working directory to find repositories
- If str: Path to directory containing git repositories
- If List[str]: List of paths to git repositories or Repository instances
ignore_repos (Optional[List[str]]): List of repository names to ignore verbose (bool, optional): Whether to print verbose output. Defaults to True. tmp_dir (Optional[str]): Directory to clone remote repositories into. Created if not provided. cache_backend (Optional[object]): Cache backend instance from gitpandas.cache default_branch (str, optional): Name of the default branch to use. Defaults to 'main'.
- repo_name()
- Returns a DataFrame containing the names of all repositories in the project.
- Returns:
- pandas.DataFrame: A DataFrame with a single column:
- •
- repository (str): Name of each repository
- is_bare()
- Returns a dataframe of repo names and whether or not they are bare.
- Returns
- DataFrame
- has_coverage()
- Returns a DataFrame of repo names and whether or not they have a .coverage file that can be parsed
- Returns
- DataFrame
- coverage()
- Will return a DataFrame with coverage information (if available) for each
repo in the project).
If there is a .coverage file available, this will attempt to form a DataFrame with that information in it, which will contain the columns:
- repository
- filename
- lines_covered
- total_lines
- coverage
If it can't be found or parsed, an empty DataFrame of that form will be returned.
- Returns
- DataFrame
- file_change_rates(branch=None, limit=None, coverage=False, days=None, ignore_globs=None, include_globs=None)
- Will return a DataFrame containing some basic aggregations of the file change history data, and optionally test coverage data from a coverage_data.py .coverage file. The aim here is to identify files in the project which have abnormal edit rates, or the rate of changes without growing the files size. If a file has a high change rate and poor test coverage, then it is a great candidate for writing more tests.
- Args:
- branch (Optional[str]): Branch to analyze. Defaults to default_branch if None. limit (Optional[int]): Maximum number of commits to return, None for no limit coverage (bool, optional): Whether to include coverage data. Defaults to False. days (Optional[int]): Number of days to return if limit is None ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore include_globs (Optional[List[str]]): List of glob patterns for files to include
- Returns:
- DataFrame: DataFrame with file change statistics and optionally coverage data
- hours_estimate(branch=None, grouping_window=0.5, single_commit_hours=0.5, limit=None, days=None, committer=True, by=None, ignore_globs=None, include_globs=None)
- Returns a DataFrame containing the estimated hours spent by each committer/author.
- Args:
- branch (Optional[str]): Branch to analyze. Defaults to default_branch if None. grouping_window (float, optional): Hours threshold for considering commits part of same session.
single_commit_hours (float, optional): Hours to assign to single commits. Defaults to 0.5. limit (Optional[int]): Maximum number of commits to analyze days (Optional[int]): If provided, only analyze commits from last N days committer (bool, optional): If True use committer, if False use author. Defaults to True. by (Optional[str]): How to group results. One of None, 'committer', 'author' ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore include_globs (Optional[List[str]]): List of glob patterns for files to include
- Returns:
- DataFrame: DataFrame with hours estimates
- commit_history(branch=None, limit=None, days=None, ignore_globs=None, include_globs=None)
- Returns a DataFrame containing the commit history for all repositories.
- Args:
- branch (Optional[str]): Branch to analyze. Defaults to default_branch if None. limit (Optional[int]): Maximum number of commits to return days (Optional[int]): If provided, only return commits from last N days ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore include_globs (Optional[List[str]]): List of glob patterns for files to include
- Returns:
- DataFrame: DataFrame with commit history
- file_change_history(branch=None, limit=None, days=None, ignore_globs=None, include_globs=None)
- Returns a DataFrame containing the file change history for all repositories.
- Args:
- branch (Optional[str]): Branch to analyze. Defaults to default_branch if None. limit (Optional[int]): Maximum number of commits to analyze days (Optional[int]): If provided, only analyze commits from last N days ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore include_globs (Optional[List[str]]): List of glob patterns for files to include
- Returns:
- DataFrame: DataFrame with file change history
- blame(committer=True, by='repository', ignore_globs=None, include_globs=None)
- Analyzes blame information across all repositories.
Retrieves blame information from the current HEAD of each repository and aggregates it based on the specified grouping. Can group results by committer/author and either repository or file.
- Args:
- committer (bool, optional): If True, group by committer name. If False, group by author name.
- Defaults to True.
- by (str, optional): How to group the results. One of:
- 'repository': Group by repository (default)
- 'file': Group by individual file
ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore include_globs (Optional[List[str]]): List of glob patterns for files to include
- Returns:
- pandas.DataFrame: A DataFrame with columns depending on the 'by' parameter:
- If by='repository':
- committer/author (str): Name of the committer/author
- loc (int): Lines of code attributed to that person
- If by='file':
- committer/author (str): Name of the committer/author
- file (str): File path
- loc (int): Lines of code attributed to that person in that file
- Note:
- Results are sorted by lines of code in descending order. If both ignore_globs and include_globs are provided, files must match an include pattern and not match any ignore patterns to be included.
- file_detail(rev='HEAD', committer=True, ignore_globs=None, include_globs=None)
- Provides detailed information about all files in the repositories.
Analyzes each file in the repositories at the specified revision, gathering information about size, ownership, and last modification.
- Args:
- rev (str, optional): Revision to analyze. Defaults to 'HEAD'. committer (bool, optional): If True, use committer info. If False, use author.
ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore include_globs (Optional[List[str]]): List of glob patterns for files to include
- Returns:
- pandas.DataFrame: A DataFrame indexed by (file, repository) with columns:
- committer/author (str): Name of primary committer/author
- last_change (datetime): When file was last modified
- loc (int): Lines of code in file
- extension (str): File extension
- directory (str): Directory containing file
- filename (str): Name of file without path
- pct_blame (float): Percentage of file attributed to primary committer/author
- Note:
- The primary committer/author is the person responsible for the most lines in the current version of the file.
- branches()
- Returns information about all branches across repositories.
Retrieves a list of all branches (both local and remote) from each repository in the project directory.
- Returns:
- pandas.DataFrame: A DataFrame with columns:
- repository (str): Repository name
- local (bool): Whether the branch is local
- branch (str): Name of the branch
- revs(branch=None, limit=None, skip=None, num_datapoints=None)
- Returns a DataFrame containing revision information for all repositories.
- Args:
- branch (Optional[str]): Branch to analyze. Defaults to default_branch if None. limit (Optional[int]): Maximum number of revisions to return skip (Optional[int]): Number of revisions to skip between samples num_datapoints (Optional[int]): If provided, evenly sample this many revisions
- Returns:
- DataFrame: DataFrame with revision information
- cumulative_blame(branch=None, by='committer', limit=None, skip=None, num_datapoints=None, committer=True, ignore_globs=None, include_globs=None)
- Returns a DataFrame containing cumulative blame information for all repositories.
- Args:
- branch (Optional[str]): Branch to analyze. Defaults to default_branch if None. by (str, optional): How to group results. Defaults to 'committer'. limit (Optional[int]): Maximum number of revisions to analyze skip (Optional[int]): Number of revisions to skip between samples num_datapoints (Optional[int]): If provided, evenly sample this many revisions committer (bool, optional): If True use committer, if False use author. Defaults to True. ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore include_globs (Optional[List[str]]): List of glob patterns for files to include
- Returns:
- DataFrame: DataFrame with cumulative blame information
- Analyze each tag, and trace backwards from the tag to all commits that make up that tag. This method looks at the commit for the tag, and then works backwards to that commits parents, and so on and so, until it hits another tag, is out of the time range, or hits the root commit. It returns a DataFrame with the branches:
- Parameters
- kwargs -- kwargs to pass to Repository.commits_in_tags
- Returns
- DataFrame
- Returns a data frame of all tags in origin. The DataFrame will have the columns:
- repository
- tag
- Returns
- DataFrame
- repo_information()
- Returns detailed metadata about each repository.
Retrieves various properties and references from each repository's Git object model.
- Returns:
- pandas.DataFrame: A DataFrame with columns:
- local_directory (str): Path to the repository
- branches (list): List of branches
- bare (bool): Whether it's a bare repository
- remotes (list): List of remote references
- description (str): Repository description
- references (list): List of all references
- heads (list): List of branch heads
- submodules (list): List of submodules
- tags (list): List of tags
- active_branch (str): Currently checked out branch
- bus_factor(ignore_globs=None, include_globs=None, by='projectd')
- Calculates the "bus factor" for the repositories.
The bus factor is a measure of risk based on how concentrated the codebase knowledge is among contributors. It is calculated as the minimum number of contributors whose combined contributions account for at least 50% of the codebase's lines of code.
- Args:
- ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore include_globs (Optional[List[str]]): List of glob patterns for files to include by (str, optional): How to calculate the bus factor. One of:
- 'projectd': Calculate for entire project directory (default)
- 'repository': Calculate separately for each repository
- 'file': Calculate separately for each file across all repositories
- Returns:
- pandas.DataFrame: A DataFrame with columns depending on the 'by' parameter:
- If by='projectd':
- projectd (str): Always 'projectd'
- bus factor (int): Bus factor for entire project
- If by='repository':
- repository (str): Repository name
- bus factor (int): Bus factor for that repository
- If by='file':
- file (str): File path
- bus factor (int): Bus factor for that file
- repository (str): Repository name
- Note:
- A low bus factor (e.g. 1-2) indicates high risk as knowledge is concentrated among few contributors. A higher bus factor indicates knowledge is better distributed.
- punchcard(branch=None, limit=None, days=None, by=None, normalize=None, ignore_globs=None, include_globs=None)
- Returns a DataFrame containing punchcard data for all repositories.
- Args:
- branch (Optional[str]): Branch to analyze. Defaults to default_branch if None. limit (Optional[int]): Maximum number of commits to analyze days (Optional[int]): If provided, only analyze commits from last N days by (Optional[str]): How to group results. One of None, 'committer', 'author' normalize (Optional[int]): If provided, normalize values to this maximum ignore_globs (Optional[List[str]]): List of glob patterns for files to ignore include_globs (Optional[List[str]]): List of glob patterns for files to include
- Returns:
- DataFrame: DataFrame with punchcard data
- bulk_fetch_and_warm(fetch_remote=False, warm_cache=False, parallel=True, remote_name='origin', prune=False, dry_run=False, cache_methods=None, **kwargs)
- Safely fetch remote changes and pre-warm cache for all repositories.
Performs bulk operations across all repositories in the project directory, optionally fetching from remote repositories and pre-warming caches to improve subsequent analysis performance.
- Args:
- fetch_remote (bool, optional): Whether to fetch from remote repositories.
- Defaults to False.
- warm_cache (bool, optional): Whether to pre-warm repository caches.
- Defaults to False.
- parallel (bool, optional): Use parallel processing when available (joblib).
- Defaults to True.
remote_name (str, optional): Name of remote to fetch from. Defaults to 'origin'. prune (bool, optional): Remove remote-tracking branches that no longer exist.
- dry_run (bool, optional): Show what would be fetched without actually fetching.
- Defaults to False.
- cache_methods (Optional[List[str]]): List of methods to use for cache warming.
- If None, uses default methods. See Repository.warm_cache for available methods.
**kwargs: Additional keyword arguments to pass to cache warming methods.
- Returns:
- dict: Results with keys:
- success (bool): Whether the overall operation was successful
- repositories_processed (int): Number of repositories processed
- fetch_results (dict): Per-repository fetch results (if fetch_remote=True)
- cache_results (dict): Per-repository cache warming results (if warm_cache=True)
- execution_time (float): Total execution time in seconds
- summary (dict): Summary statistics of the operation
- Note:
- This method safely handles errors at the repository level, ensuring that failures in one repository don't affect processing of others. All operations are read-only and will not modify working directories or current branches.
- invalidate_cache(keys=None, pattern=None, repositories=None)
- Invalidate cache entries across multiple repositories.
- Args:
- keys (Optional[List[str]]): List of specific cache keys to invalidate pattern (Optional[str]): Pattern to match cache keys (supports * wildcard) repositories (Optional[List[str]]): List of repository names to target.
- Returns:
- dict: Results with total invalidated and per-repository breakdown
- get_cache_stats()
- Get comprehensive cache statistics across all repositories.
- Returns:
- dict: Aggregated cache statistics and per-repository breakdown
Caching
Git-Pandas supports pluggable cache backends to optimize performance for expensive, repetitive operations. This is particularly useful for large repositories or when running multiple analyses.
Overview
The caching system provides: * In-memory caching for temporary results * Disk-based caching for persistent storage across sessions * Redis-based caching for distributed storage * Cache management and invalidation methods * Decorator-based caching for expensive operations * Cache timestamp tracking - know when cache entries were populated * Cache statistics and monitoring - track cache performance and usage
Available Cache Backends
In-Memory Cache (EphemeralCache)
The default in-memory cache is ephemeral and will be cleared when the process ends:
from gitpandas import Repository
from gitpandas.cache import EphemeralCache
# Create an in-memory cache with default settings
cache = EphemeralCache()
# Or customize the cache size
cache = EphemeralCache(max_keys=500)
# Use the cache with a repository
repo = Repository('/path/to/repo', cache_backend=cache)
Disk Cache (DiskCache)
For persistent caching that survives between sessions:
from gitpandas import Repository
from gitpandas.cache import DiskCache
# Create a disk cache
cache = DiskCache(filepath='/path/to/cache.gz', max_keys=1000)
# Use the cache with a repository
repo = Repository('/path/to/repo', cache_backend=cache)
Redis Cache (RedisDFCache)
For persistent caching across sessions, use Redis:
from gitpandas import Repository from gitpandas.cache import RedisDFCache # Create a Redis cache with default settings cache = RedisDFCache() # Or customize Redis connection and cache settings cache = RedisDFCache(
host='localhost',
port=6379,
db=12,
max_keys=1000,
ttl=3600 # Cache entries expire after 1 hour ) # Use the cache with a repository repo = Repository('/path/to/repo', cache_backend=cache)
Cache Timestamp Information
All cache backends now track when cache entries were populated. You can access this information without any changes to the Repository or ProjectDirectory API:
from gitpandas import Repository
from gitpandas.cache import EphemeralCache
# Create repository with cache
cache = EphemeralCache()
repo = Repository('/path/to/repo', cache_backend=cache)
# Populate cache with some operations
commit_history = repo.commit_history(limit=10)
file_list = repo.list_files()
# Check what's in the cache and when it was cached
cached_keys = cache.list_cached_keys()
for entry in cached_keys:
print(f"Key: {entry['key']}")
print(f"Cached at: {entry['cached_at']}")
print(f"Age: {entry['age_seconds']:.1f} seconds")
# Get specific cache information
key = "commit_history_main_10_None_None_None_None"
info = cache.get_cache_info(key)
if info:
print(f"Cache entry age: {info['age_minutes']:.2f} minutes")
Cache Information Methods
All cache backends support these methods for accessing timestamp information:
- list_cached_keys() - Returns list of all cached keys with metadata
- get_cache_info(key) - Returns detailed information about a specific cache entry
The returned information includes:
- cached_at - UTC timestamp when the entry was cached
- age_seconds - Age of the cache entry in seconds
- age_minutes - Age of the cache entry in minutes
- age_hours - Age of the cache entry in hours
- cache_key - The original cache key
Using the Cache Decorator
The @multicache decorator can be used to cache method results:
from gitpandas.cache import multicache @multicache(
key_prefix="method_name",
key_list=["param1", "param2"],
skip_if=lambda x: x.get("param1") is None ) def expensive_method(self, param1, param2):
# Method implementation
pass
Configuration
Cache backends can be configured with various parameters:
EphemeralCache: * max_keys: Maximum number of keys to store in memory (default: 1000)
DiskCache: * filepath: Path to the cache file (required) * max_keys: Maximum number of keys to store (default: 1000)
RedisDFCache: * host: Redis host (default: 'localhost') * port: Redis port (default: 6379) * db: Redis database number (default: 12) * max_keys: Maximum number of keys to store (default: 1000) * ttl: Time-to-live in seconds for cache entries (default: None, no expiration) * Additional keyword arguments are passed to redis.StrictRedis
Backward Compatibility
The cache timestamp functionality is fully backward compatible:
- Existing cache files will continue to work
- Old cache entries without timestamps will be automatically converted
- No changes to Repository or ProjectDirectory APIs
- All existing code continues to work unchanged
Best Practices
Shared Cache Usage
Warning:
While it's technically possible to share the same cache object across multiple Repository instances, we strongly recommend using separate cache instances for each repository for the following reasons:
Recommended Approach - Separate Caches:
from gitpandas import Repository
from gitpandas.cache import DiskCache
# Create separate cache instances for each repository
cache1 = DiskCache(filepath='repo1_cache.gz')
cache2 = DiskCache(filepath='repo2_cache.gz')
repo1 = Repository('/path/to/repo1', cache_backend=cache1)
repo2 = Repository('/path/to/repo2', cache_backend=cache2)
Benefits of Separate Caches:
- Complete Isolation: No risk of cache eviction conflicts between repositories
- Predictable Memory Usage: Each repository has its own memory budget
- Easier Debugging: Cache issues are isolated to specific repositories
- Better Performance: No lock contention in multi-threaded scenarios
- Clear Cache Management: You can clear or manage each repository's cache independently
If You Must Share Caches:
If you need to share a cache object across multiple repositories (e.g., for memory constraints), the system is designed to handle this safely:
from gitpandas import Repository
from gitpandas.cache import EphemeralCache
# Shared cache (not recommended but supported)
shared_cache = EphemeralCache(max_keys=1000)
repo1 = Repository('/path/to/repo1', cache_backend=shared_cache)
repo2 = Repository('/path/to/repo2', cache_backend=shared_cache)
# Each repository gets separate cache entries
files1 = repo1.list_files() # Creates cache key: list_files||repo1||None
files2 = repo2.list_files() # Creates cache key: list_files||repo2||None
Shared Cache Considerations:
- Repository names are included in cache keys to prevent collisions
- Cache eviction affects all repositories sharing the cache
- Memory usage is shared across all repositories
- Very active repositories may evict cache entries from less active ones
Cache Size Planning
When planning cache sizes, consider:
- Repository Size: Larger repositories generate more cache entries
- Operation Types: Some operations (like cumulative_blame) create many cache entries
- Memory Constraints: Balance cache size with available system memory
- Analysis Patterns: Frequently repeated analyses benefit from larger caches
Recommended Cache Sizes:
# Small repositories (< 1000 commits) cache = EphemeralCache(max_keys=100) # Medium repositories (1000-10000 commits) cache = EphemeralCache(max_keys=500) # Large repositories (> 10000 commits) cache = EphemeralCache(max_keys=1000) # For disk/Redis caches, you can use larger sizes cache = DiskCache(filepath='cache.gz', max_keys=5000)
API Reference
- class gitpandas.cache.DiskCache(filepath, max_keys=1000)
- Bases: EphemeralCache
An in-memory cache that can be persisted to disk using pickle.
Inherits LRU eviction logic from EphemeralCache. Thread-safe for concurrent access.
- __init__(filepath, max_keys=1000)
- Initializes the cache. Tries to load from the specified filepath if it exists.
- Parameters
- filepath -- Path to the file for persisting the cache.
- max_keys -- Maximum number of keys to keep in the cache (LRU).
- set(k, v)
- Thread-safe set operation that prevents nested save calls.
- get(k)
- Thread-safe get operation with disk loading capability.
- exists(k)
- Thread-safe exists check.
- evict(n=1)
- Thread-safe eviction.
- load()
- Loads the cache state (_cache dictionary and _key_list) from the specified filepath using pickle. Handles file not found and deserialization errors. Thread-safe operation.
- save()
- Saves the current cache state (_cache dictionary and _key_list) to the specified filepath using pickle. Creates parent directories if needed. Thread-safe operation.
- get_cache_info(k)
- Get cache entry metadata for a key.
- get_cache_stats()
- Get comprehensive cache statistics.
- Returns:
- dict: Cache statistics including size, hit rates, and age information
- invalidate_cache(keys=None, pattern=None)
- Invalidate specific cache entries or all entries.
- Args:
- keys (Optional[List[str]]): List of specific keys to invalidate pattern (Optional[str]): Pattern to match keys (supports * wildcard)
- Note:
- If both keys and pattern are None, all cache entries are invalidated.
- list_cached_keys()
- List all cached keys with their metadata.
- class gitpandas.cache.RedisDFCache(host='localhost', port=6379, db=12, max_keys=1000, ttl=None, **kwargs)
- Bases: object
A redis based cache, using redis-py under the hood.
- Parameters
- host -- default localhost
- port -- default 6379
- db -- the database to use, default 12
- max_keys -- the max number of keys to cache, default 1000
- ttl -- time to live for any cached results, default None
- kwargs -- additional options available to redis.StrictRedis
- __init__(host='localhost', port=6379, db=12, max_keys=1000, ttl=None, **kwargs)
- evict(n=1)
- set(orik, v)
- get(orik)
- exists(k)
- sync()
- Syncs the key list with what is in redis. :return: None
- get_cache_info(orik)
- Get cache entry metadata for a key.
- list_cached_keys()
- List all cached keys with their metadata.
- invalidate_cache(keys=None, pattern=None)
- Invalidate specific cache entries or all entries.
- Args:
- keys (Optional[List[str]]): List of specific keys to invalidate (without prefix) pattern (Optional[str]): Pattern to match keys (supports * wildcard, without prefix)
- Note:
- If both keys and pattern are None, all cache entries are invalidated.
- get_cache_stats()
- Get comprehensive cache statistics.
- Returns:
- dict: Cache statistics including size, hit rates, and age information
- purge()
- gitpandas.cache.multicache(key_prefix, key_list, skip_if=None)
- Decorator to cache the results of a method call.
- Args:
- key_prefix (str): Prefix for the cache key. key_list (list[str]): List of argument names (from kwargs) to include in the cache key. skip_if (callable, optional): A function that takes kwargs and returns True
The decorated method can accept an optional force_refresh=True argument to bypass the cache read but still update the cache with the new result. This force_refresh state propagates to nested calls on the same object instance.
- class gitpandas.cache.CacheEntry(data, cache_key=None)
- Bases: object
Wrapper for cached values that includes metadata.
- __init__(data, cache_key=None)
- to_dict()
- Convert to dictionary for serialization.
- classmethod from_dict(d)
- Create CacheEntry from dictionary.
- age_seconds()
- Return age of cache entry in seconds.
- age_minutes()
- Return age of cache entry in minutes.
- age_hours()
- Return age of cache entry in hours.
Remote Operations and Cache Warming
Git-pandas provides safe and efficient methods for working with remote repositories and optimizing performance through cache warming. These features allow you to keep your repositories up to date and improve analysis performance through intelligent caching.
Safe Remote Fetch
The safe_fetch_remote method allows you to safely fetch changes from remote repositories without modifying your working directory or current branch.
Repository.safe_fetch_remote()
- Repository.safe_fetch_remote(remote_name='origin', prune=False, dry_run=False)
- Safely fetch changes from remote repository.
Fetches the latest changes from a remote repository without modifying the working directory. This is a read-only operation that only updates remote-tracking branches.
- Args:
- remote_name (str, optional): Name of remote to fetch from. Defaults to 'origin'. prune (bool, optional): Remove remote-tracking branches that no longer exist on remote.
- dry_run (bool, optional): Show what would be fetched without actually fetching.
- Defaults to False.
- Returns:
- dict: Fetch results with keys:
- success (bool): Whether the fetch was successful
- message (str): Status message or error description
- remote_exists (bool): Whether the specified remote exists
- changes_available (bool): Whether new changes were fetched
- error (Optional[str]): Error message if fetch failed
- Note:
- This method is safe as it only fetches remote changes and never modifies the working directory or current branch. It will not perform any merges, rebases, or checkouts.
Basic Usage
from gitpandas import Repository
from gitpandas.cache import EphemeralCache
# Create repository with caching
cache = EphemeralCache(max_keys=100)
repo = Repository('/path/to/repo', cache_backend=cache)
# Perform a dry run to see what would be fetched
dry_result = repo.safe_fetch_remote(dry_run=True)
print(f"Would fetch from: {dry_result['message']}")
# Safely fetch changes
if dry_result['remote_exists']:
result = repo.safe_fetch_remote()
if result['success']:
print(f"Fetch completed: {result['message']}")
if result['changes_available']:
print("New changes are available!")
else:
print(f"Fetch failed: {result['error']}")
Advanced Options
# Fetch from a specific remote result = repo.safe_fetch_remote(remote_name='upstream') # Fetch and prune deleted remote branches result = repo.safe_fetch_remote(prune=True) # Perform dry run to preview without fetching result = repo.safe_fetch_remote(dry_run=True)
Safety Features
- Read-only operation: Never modifies working directory or current branch
- Error handling: Gracefully handles network errors and missing remotes
- Validation: Checks for remote existence before attempting fetch
- Dry run support: Preview operations without making changes
Cache Warming
Cache warming pre-populates the cache with commonly used data to improve performance of subsequent analysis operations.
Repository.warm_cache()
- Repository.warm_cache(methods=None, **kwargs)
- Pre-populate cache with commonly used data.
Executes a set of commonly used repository analysis methods to populate the cache, improving performance for subsequent calls. Only methods that support caching will be executed.
- Args:
- methods (Optional[List[str]]): List of method names to pre-warm. If None,
- uses a default set of commonly used methods. Available methods: - 'commit_history': Load commit history - 'branches': Load branch information - 'tags': Load tag information - 'blame': Load blame information - 'file_detail': Load file details - 'list_files': Load file listing - 'file_change_rates': Load file change statistics
- **
- kwargs: Additional keyword arguments to pass to the methods. Common arguments include: - branch: Branch to analyze (default: repository's default branch) - limit: Limit number of commits to analyze - ignore_globs: List of glob patterns to ignore - include_globs: List of glob patterns to include
- Returns:
- dict: Results of cache warming operations with keys:
- success (bool): Whether cache warming was successful
- methods_executed (List[str]): List of methods that were executed
- methods_failed (List[str]): List of methods that failed
- cache_entries_created (int): Number of cache entries created
- execution_time (float): Total execution time in seconds
- errors (List[str]): List of error messages for failed methods
- Note:
- This method will only execute methods if a cache backend is configured. If no cache backend is available, it will return immediately with a success status but no methods executed.
Basic Usage
from gitpandas import Repository
from gitpandas.cache import DiskCache
# Create repository with persistent cache
cache = DiskCache('/tmp/my_cache.gz', max_keys=200)
repo = Repository('/path/to/repo', cache_backend=cache)
# Warm cache with default methods
result = repo.warm_cache()
print(f"Cache warming completed in {result['execution_time']:.2f} seconds")
print(f"Created {result['cache_entries_created']} cache entries")
print(f"Methods executed: {result['methods_executed']}")
Custom Cache Warming
# Warm specific methods with custom parameters result = repo.warm_cache(
methods=['commit_history', 'blame', 'file_detail'],
limit=100,
branch='main',
ignore_globs=['*.log', '*.tmp'] ) # Check results if result['success']:
print(f"Successfully warmed {len(result['methods_executed'])} methods") else:
print(f"Errors occurred: {result['errors']}")
Available Methods
The following methods can be warmed:
- commit_history: Load commit history
- branches: Load branch information
- tags: Load tag information
- blame: Load blame information
- file_detail: Load file details
- list_files: Load file listing
- file_change_rates: Load file change statistics
Performance Benefits
Cache warming can significantly improve performance:
import time
# Test cold performance
start = time.time()
history_cold = repo.commit_history(limit=100)
cold_time = time.time() - start
# Warm the cache
repo.warm_cache(methods=['commit_history'], limit=100)
# Test warm performance
start = time.time()
history_warm = repo.commit_history(limit=100)
warm_time = time.time() - start
speedup = cold_time / warm_time
print(f"Cache warming provided {speedup:.1f}x speedup!")
Bulk Operations
For projects with multiple repositories, bulk operations allow you to efficiently fetch and warm caches across all repositories.
ProjectDirectory.bulk_fetch_and_warm()
- ProjectDirectory.bulk_fetch_and_warm(fetch_remote=False, warm_cache=False, parallel=True, remote_name='origin', prune=False, dry_run=False, cache_methods=None, **kwargs)
- Safely fetch remote changes and pre-warm cache for all repositories.
Performs bulk operations across all repositories in the project directory, optionally fetching from remote repositories and pre-warming caches to improve subsequent analysis performance.
- Args:
- fetch_remote (bool, optional): Whether to fetch from remote repositories.
- Defaults to False.
- warm_cache (bool, optional): Whether to pre-warm repository caches.
- Defaults to False.
- parallel (bool, optional): Use parallel processing when available (joblib).
- Defaults to True.
remote_name (str, optional): Name of remote to fetch from. Defaults to 'origin'. prune (bool, optional): Remove remote-tracking branches that no longer exist.
- dry_run (bool, optional): Show what would be fetched without actually fetching.
- Defaults to False.
- cache_methods (Optional[List[str]]): List of methods to use for cache warming.
- If None, uses default methods. See Repository.warm_cache for available methods.
**kwargs: Additional keyword arguments to pass to cache warming methods.
- Returns:
- dict: Results with keys:
- success (bool): Whether the overall operation was successful
- repositories_processed (int): Number of repositories processed
- fetch_results (dict): Per-repository fetch results (if fetch_remote=True)
- cache_results (dict): Per-repository cache warming results (if warm_cache=True)
- execution_time (float): Total execution time in seconds
- summary (dict): Summary statistics of the operation
- Note:
- This method safely handles errors at the repository level, ensuring that failures in one repository don't affect processing of others. All operations are read-only and will not modify working directories or current branches.
Basic Usage
from gitpandas import ProjectDirectory
from gitpandas.cache import DiskCache
# Create project directory with shared cache
cache = DiskCache('/tmp/project_cache.gz', max_keys=500)
project = ProjectDirectory('/path/to/repos', cache_backend=cache)
# Perform bulk operations
result = project.bulk_fetch_and_warm(
fetch_remote=True,
warm_cache=True,
parallel=True
)
print(f"Processed {result['repositories_processed']} repositories")
print(f"Fetch summary: {result['summary']['fetch_successful']} successful")
print(f"Cache summary: {result['summary']['cache_successful']} successful")
Advanced Bulk Operations
# Customize bulk operations result = project.bulk_fetch_and_warm(
fetch_remote=True,
warm_cache=True,
parallel=True,
remote_name='upstream',
prune=True,
dry_run=False,
cache_methods=['commit_history', 'blame'],
limit=200,
ignore_globs=['*.log'] ) # Check individual repository results for repo_name, fetch_result in result['fetch_results'].items():
if not fetch_result['success']:
print(f"Fetch failed for {repo_name}: {fetch_result['error']}") for repo_name, cache_result in result['cache_results'].items():
print(f"{repo_name}: {cache_result['cache_entries_created']} cache entries")
Parallel Processing
Bulk operations support parallel processing when joblib is available:
# Enable parallel processing (default when joblib available) result = project.bulk_fetch_and_warm(
fetch_remote=True,
warm_cache=True,
parallel=True # Uses all available CPU cores ) # Disable parallel processing for sequential execution result = project.bulk_fetch_and_warm(
fetch_remote=True,
warm_cache=True,
parallel=False )
Best Practices
- 1.
- Regular Fetching: Use safe_fetch_remote regularly to keep repositories current
- 2.
- Dry Run First: Use dry runs to preview fetch operations
- 3.
- Error Handling: Always check return values for errors
- 4.
- Remote Validation: Verify remotes exist before fetching
- 1.
- Persistent Caching: Use DiskCache for long-term cache persistence
- 2.
- Appropriate Cache Size: Set reasonable max_keys based on your usage
- 3.
- Selective Warming: Only warm methods you actually use
- 4.
- Regular Warming: Re-warm caches when data becomes stale
- 1.
- Shared Caches: Use shared cache backends across repositories
- 2.
- Parallel Processing: Enable parallel processing for multiple repositories
- 3.
- Custom Parameters: Tailor operations to your specific needs
- 4.
- Error Isolation: Handle errors at the repository level
Error Handling
All remote operations and cache warming methods provide comprehensive error information:
# Safe fetch error handling result = repo.safe_fetch_remote() if not result['success']:
if result['remote_exists']:
print(f"Fetch failed: {result['error']}")
else:
print(f"No remote configured: {result['message']}") # Cache warming error handling result = repo.warm_cache() if not result['success']:
print(f"Failed methods: {result['methods_failed']}")
for error in result['errors']:
print(f"Error: {error}") # Bulk operation error handling result = project.bulk_fetch_and_warm(fetch_remote=True, warm_cache=True) for repo_name, repo_result in result['fetch_results'].items():
if not repo_result['success']:
print(f"Repository {repo_name} failed: {repo_result.get('error', 'Unknown error')}")
Examples
Complete examples demonstrating these features can be found in the examples/ directory:
- examples/remote_fetch_and_cache_warming.py: Comprehensive demonstration of all features
- examples/cache_timestamps.py: Cache timestamp and metadata examples
Return Value Reference
The safe_fetch_remote method returns a dictionary with these keys:
- success (bool): Whether the fetch was successful
- message (str): Status message or description
- remote_exists (bool): Whether the specified remote exists
- changes_available (bool): Whether new changes were fetched
- error (str or None): Error message if fetch failed
The warm_cache method returns a dictionary with these keys:
- success (bool): Whether cache warming was successful
- methods_executed (list): List of methods that were executed
- methods_failed (list): List of methods that failed
- cache_entries_created (int): Number of cache entries created
- execution_time (float): Total execution time in seconds
- errors (list): List of error messages for failed methods
The bulk_fetch_and_warm method returns a dictionary with these keys:
- success (bool): Whether the overall operation was successful
- repositories_processed (int): Number of repositories processed
- fetch_results (dict): Per-repository fetch results
- cache_results (dict): Per-repository cache warming results
- execution_time (float): Total execution time in seconds
- summary (dict): Summary statistics including:
- fetch_successful (int): Number of successful fetches
- fetch_failed (int): Number of failed fetches
- cache_successful (int): Number of successful cache warming operations
- cache_failed (int): Number of failed cache warming operations
- repositories_with_remotes (int): Number of repositories with remotes
- total_cache_entries_created (int): Total cache entries created across all repositories
Performance Guide
This guide covers performance optimization strategies and best practices for using git-pandas effectively with large repositories and datasets.
Overview
Git-pandas provides several mechanisms to optimize performance:
- Caching: Multiple cache backends for reusing expensive computations
- Parallelization: Multi-threading support for repository operations
- Data Filtering: Glob patterns and limits to reduce dataset size
- Memory Management: Efficient data structures and memory usage patterns
Caching System
The caching system is the most important performance optimization in git-pandas. It stores the results of expensive Git operations and analysis computations.
Cache Backends
git-pandas supports three cache backends:
- EphemeralCache (In-Memory)
- Best for: Single-session analysis, development, testing
- Pros: Fast access, no disk I/O, automatic cleanup
- Cons: Data lost when process ends, limited by available RAM
- Use case: Interactive analysis, Jupyter notebooks
- DiskCache (Persistent)
- Best for: Multi-session analysis, CI/CD pipelines, long-running processes
- Pros: Survives process restarts, configurable size limits, compression
- Cons: Slower than memory cache, disk space usage
- Use case: Regular analysis workflows, automated reporting
- RedisDFCache (Distributed)
- Best for: Multi-user environments, distributed analysis, shared cache
- Pros: Shared across processes/machines, TTL support, Redis features
- Cons: Requires Redis server, network latency, additional complexity
- Use case: Team environments, production deployments
Cache Configuration
Basic cache setup:
from gitpandas import Repository
from gitpandas.cache import EphemeralCache, DiskCache, RedisDFCache
# In-memory cache (fastest for single session)
cache = EphemeralCache(max_keys=1000)
repo = Repository('/path/to/repo', cache_backend=cache)
# Persistent cache (best for repeated analysis)
cache = DiskCache('/tmp/gitpandas_cache.gz', max_keys=500)
repo = Repository('/path/to/repo', cache_backend=cache)
# Redis cache (best for shared environments)
cache = RedisDFCache(host='localhost', max_keys=2000, ttl=3600)
repo = Repository('/path/to/repo', cache_backend=cache)
Cache Warming
Pre-populate cache for better performance:
# Warm cache with commonly used methods result = repo.warm_cache(
methods=['commit_history', 'branches', 'blame', 'file_detail'],
limit=100, # Reasonable limit for cache warming
ignore_globs=['*.log', '*.tmp'] ) print(f"Cache entries created: {result['cache_entries_created']}") print(f"Execution time: {result['execution_time']:.2f} seconds")
Cache Management
Monitor and manage cache performance:
# Get cache statistics
stats = repo.get_cache_stats()
print(f"Repository entries: {stats['repository_entries']}")
if stats['global_cache_stats']:
global_stats = stats['global_cache_stats']
print(f"Cache usage: {global_stats['cache_usage_percent']:.1f}%")
print(f"Average entry age: {global_stats['average_entry_age_hours']:.2f} hours")
# Invalidate specific cache entries
repo.invalidate_cache(keys=['commit_history'])
# Clear old cache entries by pattern
repo.invalidate_cache(pattern='blame*')
# Clear all cache for repository
repo.invalidate_cache()
Performance Benchmarks
Typical performance improvements with caching:
| Operation | No Cache | With Cache | Speedup |
| commit_history | 2.5s | 0.05s | 50x |
| blame | 4.2s | 0.08s | 52x |
| file_detail | 1.8s | 0.03s | 60x |
| branches/tags | 0.3s | 0.01s | 30x |
| bus_factor | 3.1s | 0.06s | 51x |
Benchmarks based on medium-sized repository (~5000 commits, 500 files)
Data Filtering and Limits
Reduce dataset size to improve performance:
Glob Patterns
Use glob patterns to focus analysis on relevant files:
# Analyze only Python files commits = repo.commit_history(include_globs=['*.py']) # Exclude test and build files blame = repo.blame(ignore_globs=['test_*.py', 'build/*', '*.pyc']) # Multiple patterns rates = repo.file_change_rates(
include_globs=['*.py', '*.js', '*.html'],
ignore_globs=['*/tests/*', '*/node_modules/*'] )
Limits and Time Windows
Limit analysis scope for faster results:
# Limit to recent commits recent_commits = repo.commit_history(limit=500) # Analyze last 90 days only recent_changes = repo.file_change_rates(days=90) # Combine limits with filtering python_commits = repo.commit_history(
limit=1000,
include_globs=['*.py'] )
Branch-Specific Analysis
Analyze specific branches for better performance:
# Analyze main development branch only main_commits = repo.commit_history(branch='main') # Compare specific branches feature_commits = repo.commit_history(branch='feature/new-ui')
Parallelization
git-pandas uses parallel processing when joblib is available:
Installation
pip install joblib
Parallel Operations
Several operations automatically use parallelization:
from gitpandas import ProjectDirectory
# Parallel analysis across repositories
project = ProjectDirectory('/path/to/projects')
# These operations run in parallel when joblib is available:
commits = project.commit_history() # Parallel across repos
branches = project.branches() # Parallel across repos
blame = project.cumulative_blame() # Parallel across commits
# Control parallelization
blame = repo.parallel_cumulative_blame(
workers=4, # Number of parallel workers
limit=100 # Limit for better performance
)
Memory Management
Optimize memory usage for large repositories:
DataFrame Memory Usage
Monitor and optimize DataFrame memory:
import pandas as pd
# Check memory usage
commits = repo.commit_history(limit=10000)
print(f"Memory usage: {commits.memory_usage(deep=True).sum() / 1024 / 1024:.1f} MB")
# Optimize data types
commits['insertions'] = commits['insertions'].astype('int32')
commits['deletions'] = commits['deletions'].astype('int32')
# Use categorical for repeated strings
commits['committer'] = commits['committer'].astype('category')
Chunked Processing
Process large datasets in chunks:
def analyze_in_chunks(repo, chunk_size=1000):
"""Analyze repository in chunks to manage memory."""
all_results = []
offset = 0
while True:
# Get chunk of commits
chunk = repo.commit_history(limit=chunk_size, skip=offset)
if chunk.empty:
break
# Process chunk
result = process_chunk(chunk)
all_results.append(result)
offset += chunk_size
# Optional: Clear cache periodically
if offset % 10000 == 0:
repo.invalidate_cache(pattern='commit_history*')
return pd.concat(all_results, ignore_index=True)
Large Repository Strategies
Special considerations for large repositories:
Repository Size Guidelines
| Size | Commits/Files | Recommended Strategy |
| Small | <1K commits | Any cache, no limits |
| Medium | 1K-10K commits | DiskCache, reasonable limits |
| Large | 10K-100K commits | DiskCache + filtering |
| Very Large | >100K commits | Redis + chunking + limits |
Configuration for Large Repositories
# Large repository configuration
cache = DiskCache('/fast/disk/cache.gz', max_keys=10000)
repo = Repository(
'/path/to/large/repo',
cache_backend=cache,
default_branch='main'
)
# Use aggressive filtering
analysis = repo.commit_history(
limit=5000, # Reasonable limit
days=365, # Last year only
include_globs=['*.py', '*.js'], # Core files only
ignore_globs=['*/tests/*', '*/vendor/*'] # Exclude bulk dirs
)
Monitoring Performance
Track performance metrics:
import time def benchmark_operation(func, *args, **kwargs):
"""Benchmark any git-pandas operation."""
start_time = time.time()
start_memory = get_memory_usage()
result = func(*args, **kwargs)
end_time = time.time()
end_memory = get_memory_usage()
print(f"Execution time: {end_time - start_time:.2f}s")
print(f"Memory delta: {end_memory - start_memory:.1f}MB")
print(f"Result size: {len(result)} rows")
return result # Example usage commits = benchmark_operation(
repo.commit_history,
limit=1000,
include_globs=['*.py'] )
Performance Anti-Patterns
Avoid these common performance issues:
❌ No Caching
# Slow: No cache means repeated expensive Git operations
repo = Repository('/path/to/repo') # No cache_backend
for branch in ['main', 'develop', 'feature']:
commits = repo.commit_history(branch=branch) # Repeated work
✅ With Caching
# Fast: Cache reuses expensive operations
cache = DiskCache('/tmp/analysis_cache.gz', max_keys=1000)
repo = Repository('/path/to/repo', cache_backend=cache)
for branch in ['main', 'develop', 'feature']:
commits = repo.commit_history(branch=branch) # Cached after first
❌ No Filtering
# Slow: Processes all files including irrelevant ones blame = repo.blame() # Includes build files, logs, etc.
✅ With Filtering
# Fast: Only analyzes relevant source files blame = repo.blame(
include_globs=['*.py', '*.js', '*.html'],
ignore_globs=['*/build/*', '*/logs/*', '*.pyc'] )
❌ Unlimited Analysis
# Slow: Processes entire repository history commits = repo.commit_history() # Could be millions of commits
✅ Limited Analysis
# Fast: Focuses on recent, relevant commits commits = repo.commit_history(
limit=1000, # Last 1000 commits
days=90 # Last 90 days )
❌ Memory Leaks
# Memory issues: Large DataFrames accumulating results = [] for i in range(100):
commits = repo.commit_history(limit=10000)
results.append(commits) # Accumulating large DataFrames
✅ Memory Management
# Memory efficient: Process and aggregate total_commits = 0 for i in range(100):
commits = repo.commit_history(limit=1000) # Smaller chunks
total_commits += len(commits)
# Don't accumulate DataFrames print(f"Total commits processed: {total_commits}")
Best Practices Summary
- 1.
- Always use caching for any analysis beyond one-off queries
- 2.
- Choose the right cache backend for your use case: - EphemeralCache: Development, interactive analysis - DiskCache: Regular workflows, CI/CD - RedisDFCache: Team environments, production
- 3.
- Warm your cache before intensive analysis sessions
- 4.
- Use glob patterns to filter relevant files only
- 5.
- Set reasonable limits on commit history and time windows
- 6.
- Monitor cache performance and invalidate when needed
- 7.
- Profile memory usage for large repositories
- 8.
- Process in chunks when dealing with very large datasets
- 9.
- Use parallelization with ProjectDirectory for multiple repositories
- 10.
- Avoid anti-patterns that lead to repeated expensive operations
For more detailed examples, see the performance examples in the examples/ directory:
- examples/cache_management.py - Cache management and monitoring
- examples/remote_fetch_and_cache_warming.py - Cache warming strategies
- examples/parallel_blame.py - Parallel processing examples
Use Cases and Examples
Git-Pandas provides a powerful interface for analyzing Git repositories using pandas DataFrames. This guide demonstrates common use cases and provides practical examples.
Basic Repository Analysis
Repository Attributes
Get basic information about a repository:
from gitpandas import Repository
repo = Repository('/path/to/repo')
# Get repository name
print(repo.repo_name)
# Check if repository is bare
print(repo.is_bare())
# Get all tags
print(repo.tags())
# Get all branches
print(repo.branches())
# Get all revisions
print(repo.revs())
# Get blame information
print(repo.blame(include_globs=['*.py']))
Commit History Analysis
Analyze commit patterns and history:
# Get commit history commits_df = repo.commit_history() # Get file change history changes_df = repo.file_change_history() # Filter by file extension python_changes = repo.file_change_history(include_globs=['*.py']) # Filter by directory src_changes = repo.file_change_history(include_globs=['src/*']) # Get commits in tags tag_commits = repo.commits_in_tags()
Project-Level Analysis
Multiple Repository Analysis
Analyze multiple repositories simultaneously:
from gitpandas import ProjectDirectory # Create project from multiple repositories project = ProjectDirectory([
'git://github.com/user/repo1.git',
'git://github.com/user/repo2.git' ]) # Get repository information print(project.repo_information()) # Calculate bus factor print(project.bus_factor()) # Get file change history print(project.file_change_history()) # Get blame information print(project.blame())
Advanced Analysis
Cumulative Blame Analysis
Track code ownership over time:
# Get cumulative blame blame_df = repo.cumulative_blame() # Plot cumulative blame using pandas plotting import matplotlib.pyplot as plt blame_df.plot(x='date', y='loc', title='Cumulative Blame Over Time') plt.show()
Bus Factor Analysis
Analyze project sustainability:
# Calculate bus factor for repository bus_factor = repo.bus_factor() # Calculate file-wise bus factor (new in v2.5.0) file_bus_factor = repo.bus_factor(by='file') # Get detailed blame information blame_df = repo.blame(by='file') # Get file-level blame details # Analyze ownership patterns ownership_patterns = repo.blame(committer=True, by='repository')
Performance Optimization
Using Caching
Optimize performance with caching:
from gitpandas import Repository
from gitpandas.cache import EphemeralCache, DiskCache, RedisDFCache
# Use in-memory caching
cache = EphemeralCache(max_keys=1000)
repo = Repository('/path/to/repo', cache_backend=cache)
# Use persistent disk caching (new in v2.5.0)
disk_cache = DiskCache('/tmp/gitpandas_cache.gz', max_keys=500)
repo = Repository('/path/to/repo', cache_backend=disk_cache)
# Or use Redis for distributed caching
redis_cache = RedisDFCache(
host='localhost',
port=6379,
db=12,
ttl=3600 # Cache entries expire after 1 hour
)
repo = Repository('/path/to/repo', cache_backend=redis_cache)
Cache Management (New in v2.5.0)
Manage cache performance and contents:
# Get cache statistics
stats = repo.get_cache_stats()
print(f"Cache usage: {stats['global_cache_stats']['cache_usage_percent']:.1f}%")
# Invalidate specific cache entries
repo.invalidate_cache(keys=['commit_history'])
# Clear all cache for this repository
repo.invalidate_cache()
# Warm cache for better performance
result = repo.warm_cache(methods=['commit_history', 'blame'], limit=100)
print(f"Created {result['cache_entries_created']} cache entries")
Visualization Examples
Commit Analysis
Visualize commit patterns:
# Get commit history
commit_df = repo.commit_history()
# Plot commits over time using pandas
commit_df.resample('D').size().plot(
kind='bar',
title='Commits per Day'
)
plt.show()
File Change Analysis
Visualize file changes:
# Get file change history
changes_df = repo.file_change_history()
# Plot changes over time using pandas
changes_df.groupby('filename')['insertions'].sum().plot(
kind='bar',
title='Lines Added by File'
)
plt.show()
Best Practices
- Use caching for expensive operations like blame analysis
- Filter data early using include_globs/ignore_globs
- Leverage pandas operations for analysis
- Consider memory usage with large repositories
- Use appropriate branch names (main/master)
- Handle repository cleanup properly when using remote repositories
For more examples and detailed API documentation, see the Repository <> and Project Directory <> pages.
Contributing to Git-Pandas
We welcome contributions to Git-Pandas! Our goal is to make Git repository analysis simple, fast, and accessible to developers interested in data analysis.
Getting Started
- 1.
- Fork the repository on GitHub
- 2.
- Clone your fork locally:
git clone git@github.com:YourLogin/git-pandas.git cd git-pandas
- 3.
- Create a feature branch:
git checkout -b feature/your-awesome-feature
- 4.
- Make your changes
- 5.
- Submit a pull request
Development Guidelines
Code Style
- Follow PEP 8 style guide
- Use 4 spaces for indentation
- Maximum line length of 88 characters (Black default)
- Use snake_case for variables and functions
- Use CamelCase for classes
- Add docstrings following Google style guide
Documentation
- Write detailed docstrings for all public APIs
- Include type hints for function parameters and return values
- Document exceptions that may be raised
- Add examples where appropriate
- Update the documentation for any new features
Testing
- Write unit tests for new code
- Maintain or improve test coverage
- Run tests before submitting PR:
make test make coverage
API Design Principles
- Maintain feature parity between Repository and ProjectDirectory
- Include limit options for memory-intensive functions
- Keep the API simple and intuitive
- Consider performance implications
Current Development Focus
High Priority
- Improve test coverage with proper unit tests
- Add diff functionality between revisions
- Enhance documentation with more examples and visualizations
- Streamline documentation deployment
Recently Completed (v2.5.0)
- ✅ File-wise bus factor analysis (completed)
- ✅ Cache management methods (invalidate_cache, get_cache_stats)
- ✅ Performance documentation and optimization guide
- ✅ Remote operations (safe_fetch_remote, warm_cache, bulk operations)
- ✅ Enhanced caching system with timestamps and metadata
Feature Ideas
- Cross-branch analytics
- Enhanced verbose logging
- Language analytics and insights
- Code complexity metrics
- Contributor network analysis
Development Setup
- 1.
- Install development dependencies:
make install-dev
- 2.
- Set up pre-commit hooks:
make pre-commit
- 3.
- Run tests:
make test
- 4.
- Build documentation:
make docs
Questions?
- Open an issue for bug reports or feature requests
- Join our discussions on GitHub
- Check existing issues for similar problems
Thank you for contributing to Git-Pandas!
ADDITIONAL RESOURCES
- Index <> - Complete API reference
- Module Index <> - Module index
- Search Page <> - Search the documentation
LICENSE
This project is BSD licensed (see LICENSE.md)
Author
Will McGinnis
Copyright
2015-2025, Will McGinnis
| March 29, 2026 | 2.5.0 |
