diff --git a/.DS_Store b/.DS_Store
index 201306903d594c5815e4f6d32f437ef9cf45f97d..546acc6151e94b854cc9f637dc6c4825313be851 100644
Binary files a/.DS_Store and b/.DS_Store differ
diff --git a/Activity1/ACTIVITY1.md b/Activity1/ACTIVITY1.md
index 20e47b7fc6ed3b199a11b6145a041b100c2c053a..ed54b74f0e208dc04a6eb6f780e3786aea76e6c5 100644
--- a/Activity1/ACTIVITY1.md
+++ b/Activity1/ACTIVITY1.md
@@ -146,7 +146,7 @@ The *[queries.py](src/queries.py)* file serves as a central script for executing
#### 1. List all the players that have a contract expiring within the next 12 months
-```
+```python
def list_players_contract_expiring_next_12_months():
query = text("""
@@ -182,7 +182,7 @@ In summary, this query helps in identifying players who are nearing the end of t
#### 2. List all the female players that have had at least three years of service with the same team
-```
+```python
def list_female_players_three_years_service():
query = text("""
SELECT p.name
@@ -220,11 +220,11 @@ def list_female_players_three_years_service():
6. **HAVING Clause**:
- `HAVING COUNT(DISTINCT c.team_id) >= 1`: Ensures that the grouped results (individual players) have been associated with at least one distinct team ID in the contracts table. This clause is used to filter out any players who do not meet this criteria after the initial group by operation.
-In summary, this query identifies female players who have been with the organization for at least three years, regardless of whether they have switched teams within the organization. It ensures that only those who meet both the criteria of being with the organization for three years and having played in at least one team are included in the results.
+In summary, this query identifies female players who have been with the organisation for at least three years, regardless of whether they have switched teams within the organisation. It ensures that only those who meet both the criteria of being with the team for three years and having played in at least one team are included in the results.
#### 3. List the top 5 players in terms of income generation for the agency
-```
+```python
def list_top_5_income_generating_players():
query = text("""
SELECT p.name, SUM(c.salary_per_week * 52 * c.duration_years * c.commission_percentage / 100) AS income
@@ -263,7 +263,7 @@ In summary, this query identifies the top 5 earning players based on the sum of
#### 4. List the top 5 players (irrespective of gender) that have the longest association with the agency
-```
+```python
def list_top_5_longest_associated_players():
query = text("""
SELECT p.name, MIN(p.date_signed_up) AS signup_date
@@ -280,7 +280,7 @@ def list_top_5_longest_associated_players():
```
1. **SELECT Clause**:
- - `SELECT p.name, MIN(p.date_signed_up) AS signup_date`: Specifies that the output should include the name of the player and the earliest signup date among the entries for each player. The `MIN` function is used to find the minimum date on which each player signed up, providing insight into their initial engagement with the organization.
+ - `SELECT p.name, MIN(p.date_signed_up) AS signup_date`: Specifies that the output should include the name of the player and the earliest signup date among the entries for each player. The `MIN` function is used to find the minimum date on which each player signed up, providing insight into their initial engagement with the team.
2. **FROM Clause**:
- `FROM players p`: This specifies the primary table, `players`, from which the data about players is to be retrieved. The table is given the alias `p` to simplify referencing its columns in other parts of the query.
@@ -292,15 +292,15 @@ def list_top_5_longest_associated_players():
- `ORDER BY signup_date`: Orders the results by the earliest signup date in ascending order. This sorting allows us to see which players were the first to sign up.
5. **LIMIT Clause**:
- - `LIMIT 5`: Limits the results to the top 5 entries. This is used to fetch only the first five players who signed up with the organization.
+ - `LIMIT 5`: Limits the results to the top 5 entries. This is used to fetch only the first five players who signed up with the team.
-In summary, this query identifies the first five players who signed up with the organization by determining the earliest signup dates among all entries. It provides valuable insights into the longevity and early adopters within the player roster.
+In summary, this query identifies the first five players who signed up with the team by determining the earliest signup dates among all entries. It provides valuable insights into the longevity and early adopters within the player roster.
#### 5. Show those players that are at risk of no contract renewal, i.e. less than six months contract remaining
-```
+```python
def list_players_at_risk_no_contract_renewal():
query = text("""
SELECT p.name
diff --git a/Activity1/src/.DS_Store b/Activity1/src/.DS_Store
index e6b86e05d14cf599624456d4b115dfe74b5e3509..b37d9ce09deebbf52d37cc4420ca0af992351e96 100644
Binary files a/Activity1/src/.DS_Store and b/Activity1/src/.DS_Store differ
diff --git a/Activity2/.DS_Store b/Activity2/.DS_Store
index 54201695a6d3d4b66548bb3bf26b405a47d2f5bc..dd43e5be46077bfc0df4c5e191eecf33ac68a65e 100644
Binary files a/Activity2/.DS_Store and b/Activity2/.DS_Store differ
diff --git a/Activity2/ACTIVITY2.md b/Activity2/ACTIVITY2.md
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..57bff732c520af0f2bde4f84dde7cf6cacdecb5b 100644
--- a/Activity2/ACTIVITY2.md
+++ b/Activity2/ACTIVITY2.md
@@ -0,0 +1,383 @@
+# Advanced Databases UFCFU3-15-3 Activity 1
+## Activity 2 [Design](#design), [Implementation](#implementation), & [Queries](#queries)
+
+# Technical Specification
+
+This section outlines the technologies implemented in the development of Activity 2, an NoSQL-based database management system for a football agent managing their clients.
+
+### Hackolade
+
+**Category:** Modelling and Design Tool
+
+**Rationale for Selection:** Hackolade's supports for NoSQL databases, including MongoDB.
+
+**Role in the Project:** Utilised for the design of the database.
+#
+### Python
+
+**Category:** Programming Language
+
+**Rationale for Selection:** Simplicity, versatility, and extensive support of data manipulation and database interaction libraries.
+
+**Role in the Project:** As the core programming language for data imports, scripting, and automating data operations.
+#
+### MongoDB
+
+**Category:** NoSQL Database
+
+**Rationale for Selection:** Flexible schema, high scalability, and strong performance for large volumes of data.
+
+**Role in the Project:** Acts as the primary database system, managing large and complex data sets with efficiency and flexibility.
+#
+### PyMongo
+
+**Category:** MongoDB Driver for Python
+
+**Rationale for Selection:** Provides robust, native support for all MongoDB features through Python.
+
+**Role in the Project:** Enables Python applications to connect to MongoDB, perform data operations, and handle database management tasks efficiently.
+#
+### pandas
+
+**Category:** Data Manipulation Library
+
+**Rationale for Selection:** Powerful data structures and functions for efficient data manipulation and analysis.
+
+**Role in the Project:** Utilized for data cleaning, transformation, and analysis, facilitating data preprocessing before database insertion.
+#
+### bson
+
+**Category:** Binary JSON (BSON) Library
+
+**Rationale for Selection:** Native BSON support for MongoDB, allowing for efficient data serialization.
+
+**Role in the Project:** Used in handling MongoDB's data format, specifically for creating and managing BSON objects like ObjectId in data transactions.
+
+## Design
+
+### Document-based database model
+
+
+## Implementation
+
+The implementation of Activity 2 consists of establishing a scalable MongoDB database, focusing on flexible schema design and efficient data handling. Leveraging pymongo for MongoDB integration, the project employs Python scripts for data preprocessing with pandas, ensuring high data quality and performance. Comprehensive indexing strategies are utilised to optimise query performance. A modular codebase structure enhances the system's scalability and maintainability, facilitating the smooth integration and processing of dynamic datasets.
+
+### Activity 1 Directory Tree:
+
+```
+Activity2
+├── manage.py
+└── src
+ ├── __init__.py
+ ├── config.py
+ ├── data_import.py
+ ├── database_setup.py
+ ├── db.py
+ ├── drop_database.py
+ └── queries.py
+data
+└── players_data.csv
+```
+
+### *[config.py](src/config.py)* :
+
+Contains a single configuration variable, `DATABASE_URI`, which defines the connection string for a MongoDB database. This URI includes the database system (MongoDB) and the details necessary to connect to the MongoDB instance, typically specifying the host, port, and database name.
+
+### *[db.py](src/db.py)* :
+
+Essential for the database system's operation, this module manages the database connection. It establishes a connection using the PyMongo library, utilising the `DATABASE_URI` from *[config.py](src/config.py)*. `get_database()` returns the database object, allowing other modules to interact with the database for operations such as data retrieval, insertion, and schema management.
+
+### *[data_import.py](src/data_import.py)* :
+
+Handles the import of data into MongoDB, leveraging PyMongo for database interactions and pandas for data manipulation. `import_data()` reads data from a CSV file into a pandas DataFrame, processes the data, and inserts it into the database. Specific operations include ensuring team existence or creation in the 'teams' collection, generating MongoDB ObjectIds for new entries, and structuring player documents for insertion into the 'players' collection. Exception handling is included to manage errors during the data import process. This module is designed to work interactively with the data model defined in MongoDB, considering the dynamic nature of document-based schemas.
+
+### *[database_setup.py](src/database_setup.py)* :
+
+Responsible for setting up initial database structures, specifically focusing on creating indexes to optimise query performance. This module uses `get_database()` from *[db.py](src/db.py)* to access the MongoDB instance and then defines indexes on fields like player name, team name, and contract details within their respective collections. The script ensures that essential indexes such as uniqueness constraints on team and player names are created, enhancing data integrity and retrieval speed.
+
+### *[drop_database.py](src/drop_database.py)* :
+
+Manages the removal of data from the MongoDB database, specifically designed to facilitate a clean slate by dropping specific collections from the database. The module utilises the PyMongo library to interact with MongoDB, leveraging the database connection set up through db.py. It directly interacts with the database to drop the `contracts`, `players`, and `teams` collections.
+
+## Queries
+
+The *[queries.py](src/queries.py)* file serves as a central script for executing the 5 required SQL queries leveraging PyMongo to execute and manage database queries securely.
+
+- **Database Connection Access**:
+ - The file imports `get_database` from the *[db.py](src/db.py)* module, which provides the necessary MongoDB database object configured for accessing the database.
+
+- **Query Execution**:
+ - Each function within this file corresponds to a query that targets specific data retrieval or analysis from the database.
+ - MongoDB aggregation pipelines are used to perform complex queries. These pipelines are defined within each function, allowing for efficient data processing and transformation directly within the database.
+ - The MongoDB queries leverage operations like `$match`, `$unwind`, `$group`, `$project`, and date manipulation functions to filter, sort, and prepare data.
+
+- **Data Retrieval and Processing**:
+ - The results of each query are processed within the same function, often using list comprehensions to convert the MongoDB cursor into a list of names or other relevant data before returning.
+
+- **Main Execution Block**:
+ - The file contains an executable block that, when run directly, executes all the defined functions and prints out their results. This block demonstrates how each query function can be called and utilized to retrieve and display data.
+
+## Query Implementations
+
+#### 1. List all the players that have a contract expiring within the next 12 months
+
+```python
+def list_players_contract_expiring_next_12_months():
+ db = get_database()
+ now = datetime.now()
+ one_year_from_now = now + relativedelta(years=1)
+ pipeline = [
+ {"$match": {"contracts.start_date": {"$lte": one_year_from_now}}},
+ {"$unwind": "$contracts"},
+ {"$set": {
+ "contract_end_date": {
+ "$dateAdd": {
+ "startDate": "$contracts.start_date",
+ "unit": "year",
+ "amount": "$contracts.duration_years"
+ }
+ }
+ }},
+ {"$match": {
+ "contract_end_date": {
+ "$lt": one_year_from_now,
+ "$gte": now
+ }
+ }},
+ {"$project": {
+ "name": 1
+ }}
+ ]
+ return [doc['name'] for doc in db.players.aggregate(pipeline)]
+```
+
+1. **$match Clause**:
+ - `{"$match": {"contracts.start_date": {"$lte": one_year_from_now}}}`: This initial match filters documents in the `players` collection where any contract's start date is less than or equal to one year from now. It pre-filters players to those who have started a contract within the range that could expire within the next year.
+
+2. **$unwind Stage**:
+ - `{"$unwind": "$contracts"}`: This operation deconstructs the `contracts` array field from the input documents to output a document for each element. Each resulting document replaces the `contracts` array with the unwound document. This is necessary for processing each contract individually in subsequent stages.
+
+3. **$set Stage**:
+ - `{"$set": { "contract_end_date": {"$dateAdd": {"startDate": "$contracts.start_date", "unit": "year", "amount": "$contracts.duration_years"}}}}`: This stage calculates the end date of each contract by adding the duration of the contract, in years, to the start date. The `$dateAdd` operator is used to perform this date arithmetic directly within the database.
+
+4. **Conditional $match**:
+ - `{"$match": {"contract_end_date": {"$lt": one_year_from_now, "$gte": now}}}`: After calculating the contract end date, this match filters out the documents to include only those whose contracts end within the next year but have not yet expired. This ensures that only relevant contracts are considered.
+
+5. **$project Stage**:
+ - `{"$project": {"name": 1}}`: This stage specifies the inclusion of the `name` field and excludes all other fields from the resulting documents. It shapes the final output to focus solely on player names.
+
+6. **Return Statement**:
+ - `return [doc['name'] for doc in db.players.aggregate(pipeline)]`: This line executes the aggregation pipeline and collects the results. It constructs a list of player names from the documents returned by the aggregation query, which meets the conditions of having contracts expiring within the next year.
+
+
+In summary, this query helps in identifying players who are nearing the end of their contractual commitments within the upcoming year, providing a proactive tool for managing player contracts efficiently.
+
+#### 2. List all the female players that have had at least three years of service with the same team
+
+```python
+def list_female_players_three_years_service():
+ db = get_database()
+ now = datetime.now()
+ pipeline = [
+ {"$match": {"gender": "F"}},
+ {"$unwind": "$contracts"},
+ {"$addFields": {
+ "years_of_service": {
+ "$divide": [
+ {"$dateDiff": {
+ "startDate": "$contracts.start_date",
+ "endDate": "$$NOW",
+ "unit": "year"
+ }},
+ 1
+ ]
+ }
+ }},
+ {"$match": {"years_of_service": {"$gte": 3}}},
+ {"$group": {
+ "_id": "$_id",
+ "name": {"$first": "$name"},
+ "max_years_of_service": {"$max": "$years_of_service"}
+ }},
+ {"$match": {"max_years_of_service": {"$gte": 3}}},
+ {"$project": {"name": 1}}
+ ]
+ return [doc['name'] for doc in db.players.aggregate(pipeline)]
+```
+
+1. **$match Clause**:
+- `{"$match": {"gender": "F"}}`: This initial match filters documents in the `players` collection to include only female players. It targets players where the gender field is set to "F".
+
+2. **$unwind Stage**:
+ - `{"$unwind": "$contracts"}`: This stage deconstructs the `contracts` array field from each input document to output a document for each element. This allows subsequent operations to process each contract individually.
+
+3. **$addFields Stage**:
+ - `{"$addFields": {"years_of_service": {"$divide": [{"$dateDiff": {"startDate": "$contracts.start_date", "endDate": "$$NOW", "unit": "year"}}, 1]}}}`: This stage adds a new field, `years_of_service`, calculated by dividing the difference in years between the contract start date and the current date. This calculation determines how long each player has been under contract.
+
+4. **Conditional $match**:
+ - `{"$match": {"years_of_service": {"$gte": 3}}}`: After calculating the years of service, this match filters out the documents to include only those where the years of service are greater than or equal to 3. This ensures that only contracts meeting the minimum duration are considered.
+
+5. **$group Stage**:
+ - `{"$group": {"_id": "$_id", "name": {"$first": "$name"}, "max_years_of_service": {"$max": "$years_of_service"}}}`: This stage groups documents by player ID and aggregates the maximum years of service per player. It ensures that each player's maximum tenure is considered, even if they have had multiple contracts.
+
+6. **Final $match**:
+ - `{"$match": {"max_years_of_service": {"$gte": 3}}}`: This match ensures that only players whose maximum years of service are at least three years are included in the final result.
+
+7. **$project Stage**:
+ - `{"$project": {"name": 1}}`: This stage specifies that only the player's name should be included in the output, excluding all other fields.
+
+8. **Return Statement**:
+ - `return [doc['name'] for doc in db.players.aggregate(pipeline)]`: Executes the aggregation pipeline and constructs a list of player names who meet the criteria, which are then returned.
+
+In summary, this query identifies female players who have been with the tea, for at least three years, regardless of whether they have switched teams within the team. It ensures that only those who meet both the criteria of being with the tea, for three years and having played in at least one team are included in the results.
+
+#### 3. List the top 5 players in terms of income generation for the agency
+
+```python
+def list_top_5_income_generating_players():
+ db = get_database()
+ pipeline = [
+ {"$unwind": "$contracts"},
+ {"$addFields": {
+ "income_generation": {
+ "$multiply": ["$contracts.salary_per_week", "$contracts.duration_years", 52, "$contracts.commission_percentage", 0.01]
+ }
+ }},
+ {"$sort": {"income_generation": -1}},
+ {"$limit": 5},
+ {"$project": {"name": 1}}
+ ]
+ return [doc['name'] for doc in db.players.aggregate(pipeline)]
+```
+
+1. **$unwind Stage**:
+ - `{"$unwind": "$contracts"}`: This stage deconstructs the `contracts` array field from each input document to output a document for each element. This process allows each contract to be evaluated individually in subsequent stages.
+
+2. **$addFields Stage**:
+ - `{"$addFields": {"income_generation": {"$multiply": ["$contracts.salary_per_week", "$contracts.duration_years", 52, "$contracts.commission_percentage", 0.01]}}}`: Adds a new field called `income_generation` to each document. This field is calculated by multiplying the player's salary per week by the number of weeks in a year (52), the duration of the contract in years, and the commission percentage (converted to a decimal). This formula estimates the total income generated from each contract over its full duration.
+
+3. **$sort Stage**:
+ - `{"$sort": {"income_generation": -1}}`: This stage sorts all documents in descending order based on the `income_generation` field. The highest generating contracts come first in the order.
+
+4. **$limit Stage**:
+ - `{"$limit": 5}`: Limits the number of documents passed to the next stage to five. This effectively restricts the output to the top five highest income-generating players.
+
+5. **$project Stage**:
+ - `{"$project": {"name": 1}}`: Specifies the inclusion of the `name` field and excludes all other fields from the resulting documents. It shapes the final output to focus solely on the names of the players.
+
+6. **Return Statement**:
+ - `return [doc['name'] for doc in db.players.aggregate(pipeline)]`: Executes the aggregation pipeline and constructs a list of player names who meet the criteria of being the top income generators, which are then returned.
+
+In summary, this query identifies the top 5 earning players based on the sum of their contracted income over the duration of their contracts, taking into account their weekly salary, the number of years, and their commission percentage. It provides a powerful tool for assessing the financial impact of players in terms of their contractual agreements.
+
+#### 4. List the top 5 players (irrespective of gender) that have the longest association with the agency
+
+```python
+def list_top_5_longest_associated_players():
+ db = get_database()
+ pipeline = [
+ {"$sort": {"date_signed_up": 1}},
+ {"$limit": 5},
+ {"$project": {"name": 1}}
+ ]
+ return [doc['name'] for doc in db.players.aggregate(pipeline)]
+```
+
+1. **$sort Stage**:
+ - `{"$sort": {"date_signed_up": 1}}`: This stage sorts all documents in ascending order based on the `date_signed_up` field. The earliest signed-up players come first in the order.
+
+2. **$limit Stage**:
+ - `{"$limit": 5}`: Limits the number of documents passed to the next stage to five. This effectively restricts the output to the top five longest-associated players.
+
+3. **$project Stage**:
+ - `{"$project": {"name": 1}}`: Specifies the inclusion of the `name` field and excludes all other fields from the resulting documents. It shapes the final output to focus solely on the names of the players.
+
+4. **Return Statement**:
+ - `return [doc['name'] for doc in db.players.aggregate(pipeline)]`: Executes the aggregation pipeline and constructs a list of player names who have been associated with the club the longest, which are then returned.
+
+In summary, this query identifies the first five players who signed up with the team by determining the earliest signup dates among all entries. It provides valuable insights into the longevity and early adopters within the player roster.
+
+#### 5. Show those players that are at risk of no contract renewal, i.e. less than six months contract remaining
+
+```python
+def list_players_at_risk_no_contract_renewal():
+ db = get_database()
+ now = datetime.now()
+ six_months_from_now = now + relativedelta(months=6)
+ pipeline = [
+ {"$match": {"contracts.start_date": {"$lte": six_months_from_now}}},
+ {"$unwind": "$contracts"},
+ {"$set": {
+ "contract_end_date": {
+ "$dateAdd": {
+ "startDate": "$contracts.start_date",
+ "unit": "year",
+ "amount": "$contracts.duration_years"
+ }
+ }
+ }},
+ {"$match": {
+ "contract_end_date": {
+ "$lt": six_months_from_now,
+ "$gte": now
+ }
+ }},
+ {"$project": {
+ "name": 1
+ }}
+ ]
+ return [doc['name'] for doc in db.players.aggregate(pipeline)]
+```
+1. **$match Stage**:
+ - `{"$match": {"contracts.start_date": {"$lte": six_months_from_now}}}`: This initial match filters documents to include only those contracts that started on or before six months from now, targeting players whose contracts might be nearing expiry.
+
+2. **$unwind Stage**:
+ - `{"$unwind": "$contracts"}`: This stage deconstructs the `contracts` array field from each input document to output a document for each element. This allows each contract to be evaluated individually in subsequent stages.
+
+3. **$set Stage**:
+ - `{"$set": {"contract_end_date": {"$dateAdd": {"startDate": "$contracts.start_date", "unit": "year", "amount": "$contracts.duration_years"}}}}`: Calculates the end date of each contract by adding the duration of the contract to the start date.
+
+4. **Conditional $match**:
+ - `{"$match": {"contract_end_date": {"$lt": six_months_from_now, "$gte": now}}}`: This match filters out the documents to include only those whose contracts end within the next six months but have not yet expired, indicating they are at risk of not being renewed.
+
+5. **$project Stage**:
+ - `{"$project": {"name": 1}}`: Specifies the inclusion of the `name` field and excludes all other fields from the resulting documents. It shapes the final output to focus solely on the names of the players at risk.
+
+6. **Return Statement**:
+ - `return [doc['name'] for doc in db.players.aggregate(pipeline)]`: Executes the aggregation pipeline and constructs a list of player names at risk of not having their contracts renewed, which are then returned.
+
+
+In summary, this query identifies players whose contracts are currently active but will expire within the next six months. It is useful for flagging contracts that need attention for renewal discussions or termination preparations.
+
+## Query Results
+
+```
+Players with contracts expiring in the next 12 months:
+Name: OB001
+Name: OB022
+Name: OG005
+Name: NG032
+Name: NG222
+
+Female players with at least three years of service with the same team:
+Name: NG032
+Name: OG005
+
+Top 5 income-generating players:
+Name: NB009
+Name: NB311
+Name: OB124
+Name: OB022
+Name: NB212
+
+Top 5 longest associated players:
+Name: OG005
+Name: NG032
+Name: OB022
+Name: NB337
+Name: NG001
+
+Players at risk of no contract renewal:
+Name: OB022
+Name: NG032
+```
\ No newline at end of file
diff --git a/Activity2/images/Football-Agent.png b/Activity2/images/Football-Agent.png
new file mode 100644
index 0000000000000000000000000000000000000000..3fae68bd2a7306cc6c1589e64f54300048ee7207
Binary files /dev/null and b/Activity2/images/Football-Agent.png differ
diff --git a/Activity2/src/.DS_Store b/Activity2/src/.DS_Store
index 7f4e7d84b1b57c34e0a8f9570ad321917c38cb5f..c8865e6586eed9c30ef0ce988f505879453c5461 100644
Binary files a/Activity2/src/.DS_Store and b/Activity2/src/.DS_Store differ