Data Shield System is a Python-based automated file backup and synchronization tool designed to monitor directories and maintain updated copies of important files.
The system uses MD5 checksum comparison to detect newly created and modified files, automatically synchronizes them with a specified backup directory, and generates timestamped log reports for every backup operation.
By combining Python's file-handling capabilities, hashing algorithms, and automated task scheduling, Data Shield reduces repetitive manual backup operations and provides an efficient approach to directory-level data management.
-
Automated File Backup: Automatically copies files from a source directory to a designated backup directory at configurable time intervals.
-
MD5 Checksum-Based Change Detection: Generates MD5 checksums to compare source and destination files and identify modifications.
-
New File Detection: Identifies newly added files and automatically copies them to the backup destination.
-
Modified File Detection: Detects changes in existing files by comparing their MD5 checksums and updates the corresponding backup copies.
-
Recursive Directory Traversal: Scans nested directories while maintaining the original folder structure in the backup destination.
-
Automated Task Scheduling: Uses Python's Schedule library to execute backup operations periodically without requiring manual intervention for every backup.
-
Detailed Logging System: Generates timestamped log reports containing backup details, detected files, copying errors, and synchronization statistics.
-
Command-Line Interface: Provides a simple CLI with configurable backup intervals, source and destination paths, and built-in help instructions.
Data Shield follows a systematic backup and synchronization workflow.
The user provides three command-line arguments:
- Time Interval: Frequency of backup execution in minutes.
- Source Directory: Folder containing the original files.
- Destination Directory: Folder where backup copies are maintained.
The Python Schedule library schedules the backup function to execute repeatedly at the specified time interval.
The application uses Python's os.walk() function to recursively traverse the source directory and identify files within all nested folders.
The application performs two types of checks:
New File Detection
- Checks whether a corresponding file exists in the destination directory.
- If the file does not exist, it is identified as a new file.
- The new file is copied to the backup destination.
Modified File Detection
- Calculates the MD5 checksum of the source file.
- Calculates the MD5 checksum of its corresponding destination file.
- Compares both checksums.
- If the checksums differ, the destination file is updated.
The application uses shutil.copy2() to copy new or modified files while preserving available file metadata.
The original directory structure is maintained in the destination directory.
After every backup cycle, the application generates a timestamped log file containing:
- Backup execution date and time.
- Newly detected and copied files.
- Modified files that were updated.
- Errors encountered during file copying.
- Total number of files synchronized.
After completing a backup cycle, the scheduler waits for the next configured interval and repeats the process.
The following screenshot demonstrates the application running through the command-line interface, initializing the scheduler, and executing the backup process.
The application generates detailed log reports showing newly detected files, successful synchronization operations, and the total number of files backed up.
| Technology / Library | Purpose |
|---|---|
| Python 3 | Core programming language |
| os | Directory traversal and file management |
| sys | Command-line argument handling |
| time | Time intervals and timestamps |
| hashlib | MD5 checksum generation |
| shutil | File copying and metadata preservation |
| schedule | Periodic task automation |
Data-Shield-System/
โ
โโโ DataShieldSystem.py # Main automation script
โ
โโโ requirements.txt # Project dependencies
โ
โโโ .gitignore # Git ignored files
โ
โโโ README.md # Project documentation
โ
โโโ LICENSE # MIT License
โ
โโโ screenshots/
โโโ backup.png # Backup execution screenshot
โโโ log-report.png # Generated backup report
Follow these steps to run Data Shield System on your local machine.
- Python 3.x installed.
- pip package manager.
- Basic knowledge of command-line operations.
git clone https://github.com/bemo-codes/Data-Shield-System.gitNavigate to the project directory:
cd Data-Shield-Systempython -m venv venvActivate the virtual environment.
Windows:
venv\Scripts\activateLinux / macOS:
source venv/bin/activatepip install -r requirements.txtThe project requires the following external Python package:
schedule
Other modules used in the application are included in Python's standard library.
Data Shield operates through a command-line interface.
To view the functionalities supported by the application:
python DataShieldSystem.py --hTo understand the required command-line arguments:
python DataShieldSystem.py --uUse the following command format:
python DataShieldSystem.py <Time_Interval> <Source_Folder> <Destination_Folder>Example (Windows):
python DataShieldSystem.py 1 "C:\Users\YourName\Documents" "D:\DataShieldBackup"This command configures the application to:
- Monitor the Documents directory.
- Execute a backup operation every 1 minute.
- Detect newly created and modified files.
- Synchronize detected changes with the backup directory.
- Generate timestamped backup log reports.
Important:
- The first scheduled backup runs after the configured interval, not immediately.
- The application must remain running for scheduled backups to continue.
- Press
Ctrl + Cto stop the automation script. - Keep the destination directory outside the source directory to avoid scanning backup files recursively.
- The parent directory of the destination must already exist if the destination folder itself needs to be created.
Data Shield automatically generates a separate log file for every backup cycle.
DataShieldBackup/
โ
โโโ BackupLog_2026-11-05_21-10-45.txt
โโโ BackupLog_2026-11-05_21-15-45.txt
โ
โโโ Mumbai/
โ โโโ File_Create_Close.py
โ โโโ File_Create_Write.py
โ
โโโ Pune/
โ โโโ File_read.py
โ
โโโ Satara/
โโโ AutomationScript1.py
--------------------------------------------------
Data Shield: Backup & Sync Report
Backup executed at: 2026-11-05_21-15-43
--------------------------------------------------
New File Detected & Copied: Mumbai\File_Create_Close.py
New File Detected & Copied: Mumbai\File_Create_Exception.py
New File Detected & Copied: Pune\File_read.py
New File Detected & Copied: Satara\AutomationScript1.py
--------------------------------------------------
Total files synchronized/backed up: 14
--------------------------------------------------
End of log file
--------------------------------------------------
The actual log entries depend on the files detected during each backup cycle.
This project demonstrates the practical implementation of several Python programming concepts:
- Reading files in binary mode.
- Opening and closing files.
- Copying files between directories.
- Checking directory existence.
- Creating directories.
- Recursive directory traversal.
- Maintaining relative folder structures.
- Generating MD5 checksums using
hashlib. - Comparing file hashes.
- Detecting file modifications through checksum comparison.
- Scheduling recurring tasks.
- Executing scheduled functions.
- Managing continuous execution loops.
- Handling file access errors.
- Managing file copying exceptions.
- Handling invalid command-line arguments.
- Generating timestamped log files.
- Recording backup operations.
- Tracking synchronization counts and errors.
- The application performs periodic directory scanning rather than real-time filesystem event monitoring.
- It detects new and modified files but does not delete destination files when their source counterparts are removed.
- MD5 is used for file change detection, not cryptographic security or protection against deliberate file tampering.
- Backup files are not encrypted.
- The application does not currently provide file restoration functionality.
- The Python script must remain running for scheduled execution to continue.
- The destination directory should be separate from the source directory.
Potential improvements for future versions include:
- Email Notifications: Send notifications after successful or failed backup operations.
- Incremental Backup Optimization: Improve backup efficiency by processing only necessary file changes.
- File Restoration: Allow users to restore previously backed-up files.
- Backup Compression: Compress backup directories using ZIP archives.
- GUI Dashboard: Introduce a graphical interface for monitoring backup operations.
- SHA-256 Integration: Introduce stronger hashing algorithms for integrity verification.
- Configurable Exclusions: Allow users to exclude specific file types and folders.
- Cloud Storage Integration: Support remote backup destinations.
- Advanced Error Handling: Improve recovery mechanisms for failed backup operations.
Shivam Ramesh Kurlekar
GitHub: @bemo-codes
Project Repository: Data Shield System
This project is licensed under the MIT License.
See the LICENSE file for more information.
Built with Python ๐ | Automated for Reliability ๐ก๏ธ

