From 0a9689d5e3dd39eef1b9011a200e13d477b8eb0e Mon Sep 17 00:00:00 2001 From: J-onasJones Date: Wed, 9 Oct 2024 18:57:46 +0200 Subject: [PATCH] added turbo-octo-potato docs --- docs/turbo-octo-potato/intro.mdx | 17 + docs/turbo-octo-potato/rpopfetch.mdx | 26 ++ .../spotify-scripts/_category_.json | 10 + .../spotify-scripts/discographymaker.mdx | 58 +++ .../spotify-scripts/intro_outro_maker.mdx | 31 ++ .../spotify-scripts/likedsongsync2.mdx | 33 ++ .../steam_achievement_completion.mdx | 22 + docs/turbo-octo-potato/top_lib.mdx | 410 ++++++++++++++++++ 8 files changed, 607 insertions(+) create mode 100644 docs/turbo-octo-potato/intro.mdx create mode 100644 docs/turbo-octo-potato/rpopfetch.mdx create mode 100644 docs/turbo-octo-potato/spotify-scripts/_category_.json create mode 100644 docs/turbo-octo-potato/spotify-scripts/discographymaker.mdx create mode 100644 docs/turbo-octo-potato/spotify-scripts/intro_outro_maker.mdx create mode 100644 docs/turbo-octo-potato/spotify-scripts/likedsongsync2.mdx create mode 100644 docs/turbo-octo-potato/steam_achievement_completion.mdx create mode 100644 docs/turbo-octo-potato/top_lib.mdx diff --git a/docs/turbo-octo-potato/intro.mdx b/docs/turbo-octo-potato/intro.mdx new file mode 100644 index 0000000..5aa6106 --- /dev/null +++ b/docs/turbo-octo-potato/intro.mdx @@ -0,0 +1,17 @@ +--- +sidebar_position: 1 +slug: /turbo-octo-potato +--- + +# Turbo-Octo-Potato +*The repo that holds it all!* + +This is the repository where I dump all my random scripts that I write fro automating simple tasks. Find the repo on [Github](https://github.com/JonasunderscoreJones/turbo-octo-potato). + +## Table of Contents +The scripts for which I have a documentation are listed below: +- [TOP Lib](/turbo-octo-potato/top_lib) +- [Tabletop Achievements](/tabletop_achievement_scripts) +- [Spotify Scripts](/category/spotify-scripts) +- [Steam Achievement Completion](/turbo-octo-potato/steam_achievement_completion) +- [r/kpop Comebacks Fetcher](/turbo-octo-potato/rpopfetch) diff --git a/docs/turbo-octo-potato/rpopfetch.mdx b/docs/turbo-octo-potato/rpopfetch.mdx new file mode 100644 index 0000000..ae9fbdf --- /dev/null +++ b/docs/turbo-octo-potato/rpopfetch.mdx @@ -0,0 +1,26 @@ +# r/kpop Comebacks Fetcher +View this script on [GitHub](https://github.com/JonasunderscoreJones/turbo-octo-potato/blob/main/rpopfetch.py) + +This script fetches the K-Pop comebacks (new releases) data from the `r/kpop` subreddit wiki and converts it to json code. It also has an option to upload the json file using `rclone` + +## Setup +The script requires a few environment variables to be set. For this, populate the `.env` file with the following variables: +```properties title="../.env" +REDDIT_CLIENT_ID=[your reddit client ID] +REDDIT_CLIENT_SECRET=[your reddit client secret] +REDDIT_USER_AGENT=[your reddit user agent] +``` +For more info on how to get the `client id` and `client secret` go to the [Reddit API Docs](https://www.reddit.com/prefs/apps) + +## Usage +Inside the repository directory run the script by executing: +```bash +python rpopfetch.py [commandline arguments] +``` + +The script will output a json file with the data in the same directory as the script. + +You will also have the option to add commandline arguments: +- Verbose Logging (`--verbose` or `-v`): This will print out more information about what the script is doing +- Upload to rclone (`--upload` or `-u`): This will upload the json file to the specified rclone remote and path. The remote and path are specified in the script itself. + diff --git a/docs/turbo-octo-potato/spotify-scripts/_category_.json b/docs/turbo-octo-potato/spotify-scripts/_category_.json new file mode 100644 index 0000000..0ebe0eb --- /dev/null +++ b/docs/turbo-octo-potato/spotify-scripts/_category_.json @@ -0,0 +1,10 @@ +{ + "position": 2, + "label": "Spotify Scripts", + "collapsible": true, + "collapsed": true, + "link": { + "type": "generated-index", + "description": "Documentation for the Spotify scripts" + } + } \ No newline at end of file diff --git a/docs/turbo-octo-potato/spotify-scripts/discographymaker.mdx b/docs/turbo-octo-potato/spotify-scripts/discographymaker.mdx new file mode 100644 index 0000000..91df6f1 --- /dev/null +++ b/docs/turbo-octo-potato/spotify-scripts/discographymaker.mdx @@ -0,0 +1,58 @@ +# Discography Maker +View this script on [GitHub](https://github.com/JonasunderscoreJones/turbo-octo-potato/blob/main/spotify_scripts/discographymaker.py) + +This script fetches all songs of all albums/EP's/Singles of the artists you choose and puts them in chronological order into a playlist. + +## Setup +The script requires a few environment variables to be set. For this, populate the `.env` file with the following variables: +```properties title="../.env" +SPOTIFY_CLIENT_ID=[your spotify client ID] +SPOTIFY_CLIENT_SECRET=[your spotify client secret] +SPOTIFY_REDIRECT_URI=[your spotifiy redirect URI] +SPOTIFY_USER_ID=[your spotify user id] +``` +For more info on how to get the `client id`, `cliene secret` and `redirect uri` visit the [Spotify API Docs](https://developer.spotify.com/documentation/web-api#getting-started) for creating an app + +## Usage +Inside the repository directory run the script by executing: +```bash +python spotify_scripts/discographymaker.py +``` +You will be asked to enter the names (one at a time) of the artists You would like to be in a new discography playlist. +```bash +Authenticating... +Authenticated! +Input your first Artist (name, spotify ID or artist link). Enter other's when prompted next: +``` +If Your input matches the found artist's name perfectly, then You won't have to confirm it: +```bash +Input your first Artist (name, spotify ID or artist link). Enter other's when prompted next: +ATEEZ +Looking up Artist... +====================== +Input your next Artist (name or Spotify ID). Leave empty if all have been inputed +``` +If the found artist's name differs, You will be asked for comfirmation: +```bash +Input your next Artist (name or Spotify ID). Leave empty if all have been inputed +ateez +Looking up Artist... + The Artist doesn't exist on Spotify. Did you mean "ATEEZ" [ENTER for Yes, No for retry]: +``` +In case it found a different artist, You can choose to retry the search. + +Alternatively the scripts also allows inputs for the artist link or ID. + +:::note + +The spotify search by name feature isn't always that accurate so pasting the direct artist link or ID into the input is sometimes the only option. + +::: + +:::info + +Because of a bug in the Spotify API, if an artist has more than 50 albums/EP's/Singles, the script will only fetch the first 50. + +I am unable to fix this since it's a bug on Spotify's end and has existed for quite a while :/ + +::: diff --git a/docs/turbo-octo-potato/spotify-scripts/intro_outro_maker.mdx b/docs/turbo-octo-potato/spotify-scripts/intro_outro_maker.mdx new file mode 100644 index 0000000..47551a8 --- /dev/null +++ b/docs/turbo-octo-potato/spotify-scripts/intro_outro_maker.mdx @@ -0,0 +1,31 @@ +# Intro Outro Playlist Maker +View this script on [GitHub](https://github.com/JonasunderscoreJones/turbo-octo-potato/blob/main/spotify_scripts/intro_outro_maker.py) + +The script fetches all intro's and outro's from all artists You follow on spotify and puts them into a playlist. + +## Setup +The script requires a few environment variables to be set. For this, populate the `.env` file with the following variables: +```properties title="../.env" +SPOTIFY_CLIENT_ID=[your spotify client ID] +SPOTIFY_CLIENT_SECRET=[your spotify client secret] +SPOTIFY_REDIRECT_URI=[your spotifiy redirect URI] +SPOTIFY_USER_ID=[your spotify user id] +INTROOUTROPLAYLIST_ID=[your spotify playlist ID (create this playlist beforehand)] +``` +For more info on how to get the `client id`, `client secret` and `redirect uri` visit the [Spotify API Docs](https://developer.spotify.com/documentation/web-api#getting-started) for creating an app + +## Usage +Inside the repository directory run the script by executing: +```bash +python spotify_scripts/intro_outro_maker.py +``` + +:::warning + +This script has the potential to get your account temporarily rate limited by Spotify. + +In case of a ratelimit, the script will crash. Depending on how many artists you follow, the script might crash before finishing. + +Rate limits are not to worry about, they are temporary and will be lifted after an hour at most. + +::: diff --git a/docs/turbo-octo-potato/spotify-scripts/likedsongsync2.mdx b/docs/turbo-octo-potato/spotify-scripts/likedsongsync2.mdx new file mode 100644 index 0000000..4e754db --- /dev/null +++ b/docs/turbo-octo-potato/spotify-scripts/likedsongsync2.mdx @@ -0,0 +1,33 @@ +# Liked Song Sync +View this script on [GitHub](https://github.com/JonasunderscoreJones/turbo-octo-potato/blob/main/spotify_scripts/likedsongsync2.py) + +The script does two things with your liked songs on spotify: +- Adds them to a playlist that can be made public and displayed on your profile (unlike the default liked songs playlist) +- Syncs the liked songs to your [Last.fm](https://www.last.fm/) account + +## Setup +The script requires a few environment variables to be set. For this, populate the `.env` file with the following variables: +```properties title="../.env" +SPOTIFY_CLIENT_ID=[your spotify client ID] +SPOTIFY_CLIENT_SECRET=[your spotify client secret] +SPOTIFY_REDIRECT_URI=[your spotifiy redirect URI] +SPOTIFY_USER_ID=[your spotify user id] +LASTFM_API_KEY=[your last.fm API key] +LASTFM_API_SECRET=[your last.fm API secret] +LASTFM_USERNAME=[your last.fm username] +LASTFM_PASSWORD_HASH=[your last.fm password hash (use an online hasher to hash the password)] +LIKEDSONGPLAYLIST_ID=[your spotify playlist ID (create this playlist beforehand)] +``` +For more info on how to get the `client id`, `client secret` and `redirect uri` visit the [Spotify API Docs](https://developer.spotify.com/documentation/web-api#getting-started) for creating an app + +For more info on how to get the `API key` and `API secret` visit the [Last.fm API Docs](https://www.last.fm/api/authentication) for applying for a key. + +## Usage +Inside the repository directory run the script by executing: +```bash +python spotify_scripts/likedsongsync2.py [commandline arguments] +``` +The script has the following commandline arguments: +- Verbose Logging (`--verbose` or `-v`): This will print out more information about what the script is doing +- Force Resync: (`--force-all` or `-f`): This will force the script to resync all liked songs to Last.fm, even if they have been synced before. By default the script will only sync new liked songs. +- Skip Song Sync: (`--skip-sync` or `-s`): This will skip the syncing of a specific song. diff --git a/docs/turbo-octo-potato/steam_achievement_completion.mdx b/docs/turbo-octo-potato/steam_achievement_completion.mdx new file mode 100644 index 0000000..093779c --- /dev/null +++ b/docs/turbo-octo-potato/steam_achievement_completion.mdx @@ -0,0 +1,22 @@ +# Steam Achievement Completion +View this script on [GitHub](https://github.com/JonasunderscoreJones/turbo-octo-potato/blob/main/steam_achievement_completion.py) + +This script gathers the achievement completion rate of each game of your steam library and outputs the average game completion rate. + +## Setup +1. Install the required packages by running: +```bash +pip install requests os dotenv +``` +2. Create a `.env` file in the same directory as the script and add the following: +```properties +STEAM_API_KEY=YOUR_API_KEY +STEAM_USER_ID=YOUR_STEAM_ID +``` +Go to [Steam API](https://steamcommunity.com/dev/apikey) to get your API key and find your Steam ID by going to your profile and copying the number at the end of the URL. + +## Usage +Run the script by executing: +```bash +python steam_achievement_completion.py +``` \ No newline at end of file diff --git a/docs/turbo-octo-potato/top_lib.mdx b/docs/turbo-octo-potato/top_lib.mdx new file mode 100644 index 0000000..449c242 --- /dev/null +++ b/docs/turbo-octo-potato/top_lib.mdx @@ -0,0 +1,410 @@ +# TOP Python Library +Visit the code on [GitHub](https://github.com/JonasunderscoreJones/turbo-octo-potato/blob/main/top_lib.py) + +This is the library that holds the following code: +- [Spotify and Last.fm Authentication](#spotify-and-lastfm-authentication) +- [Spotify API manager and helper](#spotify-manager) +- Progressbar code e.g. for the [likedsongsync2.py](/turbo-octo-potato/spotify-scripts/likedsongsync2) script + +## Credentials and Setup + +The library makes use of the following environment variables defined in the `.env` file: +```properties +LASTFM_API_KEY = "lastfm_api_key" +LASTFM_API_SECRET = "lastfm_api_secret" +LASTFM_USERNAME = "lastfm_username" +LASTFM_PASSWORD_HASH = "lastfm_password_hash" + +SPOTIFY_CLIENT_ID = "spotify_client_id" +SPOTIFY_CLIENT_SECRET = "spotify_client_secret" +SPOTIFY_REDIRECT_URI = "http://localhost:6969" +SPOTIFY_USER_ID = "spotify_user_id" + +LIKEDSONGPLAYLIST_ID = "spotify_playlist_id" +SOMEPLAYLIST_ID = "spotify_playlist_id" +INTROOUTROPLAYLIST_ID = "spotify_playlist_id" +RANDOMTESTPLAYLIST_ID = "spotify_playlist_id" + +GITHUB_API_TOKEN = "github_api_token" + +REDDIT_CLIENT_ID = "reddit_client_id" +REDDIT_CLIENT_SECRET = "reddit_client_secret" +REDDIT_USER_AGENT = "reddit_user_agent" + +DISCORD_WEBHOOK_URL = 'discord_webhook_url' +``` +:::info[Note] + +Depending on what you want to use the library for, you might not need all of these environment variables. + +::: + +## Spotify and Last.fm Authentication +The library has a class called `Auth` that handles the authentication for both Spotify and Last.fm. + +### Creating an Authentication Instance +A new authentication instance can be created by calling the class with the following arguments: + +```python +my_auth = Auth( + verbose:bool, + lastfm_network:pylast.LastFMNetwork, + spotify:spotipy.Spotify, +) +``` +- `verbose` is an _optional_ boolean that determines if the authentication process should print out more information. +- `lastfm_network` and `spotify` are _optional_ arguments that can be provided if the authentication for those services has already been done. + +:::tip[Example] + +Therefore if we don't have the authentication for Last.fm and Spotify yet and we want verbose logging enabled, we can create a new instance like this: + +```python +my_auth = Auth( + verbose=True +) +``` + +::: + +### Authenticate Spotify +Our newly created authentication instance can be used to authenticate Spotify by calling the `newSpotifyauth()` method: + +```python +my_auth.newSpotifyauth( + scope:str +) +``` +- `scope` is a _required_ string that defines the permissions the Spotify API should have. For more information on the scopes visit the [Spotify API Docs](https://developer.spotify.com/documentation/general/guides/scopes/) + +This automates the authentication process as much as possible. On first launch, the user is still required to authorize the script to access the spotify API. + +### Authenticate Last.fm +Much like the spotify instance, we can authenticate Last.fm by calling the `newLastfmauth()` method: + +```python +my_auth.newLastfmauth() +``` + +This will open a browser window where the user can authorize the script to access their Last.fm account on first launch. + +### Verbose Logging +If verbose logging was enabled when creating the instance, the authentication process will print out more information about what it's doing. + +Verbose logging can be toggled at any time using the `verbose()` method: + +```python +my_auth.verbose( + verbose:bool +) +``` +- `verbose` is a _required_ boolean that determines if the authentication process should print out more information. + +### Getting the Spotify and Last.fm Instances +After the authentication process has been completed, the Spotify and Last.fm instances can be accessed using the `getSpotify()` and `getLastfm()` methods although there are a bunch of helper and automatioon functions in the library that dont require you to get the instances manually e.g. using the [Spotify Manager](#spotify-manager) class.: + +```python +spotify_instance = my_auth.getSpotify() + +lastfm_instance = my_auth.getLastfm() +``` + +### Get the used Credentials +The credentials used for the authentication can be accessed using the `getCredentials()` method: + +```python +credentials = my_auth.getCredentials() +``` + +the credentials are stored in a dictionary with the following keys: +- `LASTFM_API_KEY` +- `LASTFM_API_SECRET` +- `LASTFM_USERNAME` +- `LASTFM_PASSWORD_HASH` +- `SPOTIFY_CLIENT_ID` +- `SPOTIFY_CLIENT_SECRET` +- `SPOTIFY_REDIRECT_URI` +- `SPOTIFY_USER_ID` + +The keys are self-explanatory and contain the credentials used for the authentication. + + +## Spotify Manager +The library has a class called `SpotifyManager` that helps with managing more complex interactions with the Spotify API. + +### Creating a Spotify Manager Instance +A new Spotify Manager instance can be created by calling the class with the following arguments: + +```python +my_spotify_manager = SpotifyManager( + spotify:spotipy.Spotify +) +``` +:::tip[Example] + +We can plug in the newly created authenticated Spotify instance from above like this: + +```python +my_spotify_manager = SpotifyManager( + auth.getSpotify() +) +``` +::: + +### Get an artist's Albums +The `fetchArtistAlbums()` method can be used to get all the albums of a specific artist: + +```python +albums = my_spotify_manager.fetchArtistAlbums( + artist:str, + raise_error:bool +) +``` +- `artist` is the artist_id of the artist we want to get the albums from. +- `raise_error` is an _optional_ boolean that determines if the method should raise an error if the artist has more than 50 albums/EP's/Singles. + +:::info + +The Spotify API has a bug where it only fetches the first 50 albums/EP's/Singles of an artist if they have more than 50. This is a bug on Spotify's end and has existed for quite a while. + +::: + +### Get an Album's Tracks +The `getTrackUrisFromAlbum()` method can be used to get all the tracks of a specific album: + +```python +tracks = my_spotify_manager.getTrackUrisFromAlbum( + album:str +) +``` +- `album` is the album_id of the album we want to get the tracks from. + +The method returns a list of track uris that can be used to add the tracks to a playlist. + +:::tip[Example] + +You can also lookup the name or other info about a track. + +In this example, let's get the tracks of the album "The Story of Light" by SHINee and print the name of the first track: + +```python +# Get the album's tracks +tracks = my_spotify_manager.getTrackUrisFromAlbum( + "spotify:album:1zK5C9xg5Fz3J0bG6VwQFv" +) + +# Get the Spotify instance +spotify = auth.getSpotify() + +# Get the first track +track = spotify.track(tracks[0]) + +# Print the track's name +print(track["name"]) +``` +::: + +### Get a user's followed Artists +The `fetchUserFollowedArtists()` method can be used to get all the artists a user follows: + +```python +artists = my_spotify_manager.fetchUserFollowedArtists() +``` +The method returns the artist ids and names of all followed artists as tuples in a list: + +```python +[ + ("spotify:artist:1dfeR4HaWDbWqFHLkxsg1d", "SHINee"), + ("spotify:artist:0C8C8YiEiJqfI5fSG5Z6Y2", "ATEEZ"), + ... +] +``` + + +## Progressbar +The library has a class called `Progressbar` that helps with creating progress bars for scripts that take a long time to run. + +### Creating a Progressbar Instance +A new Progressbar instance can be created by calling the class with the following arguments: + +```python +my_progressbar = Progressbar( + total:int, + etaCalc: : Callable[[int, int], int] +) +``` +- `total` is the total number of steps the progress bar should have. +- `etaCalc` is an _optional_ function that calculates the estimated time of completion for the progress bar. + +:::tip[Example] +Let's create a new progress bar with 100 steps and a simple ETA calculation function: + +```python +# the eta_calculator function takes a current step and the total number of steps and returns an estimated time of completion in seconds +def eta_calculator(current:int, total:int) -> int: + return (total - current) * 0.1 # simple calculator that estimates 0.1 seconds per step + +my_progressbar = Progressbar( + 100, + eta_calculator +) +``` +::: + +The total number of steps can be updated at any time using the `setTotal()` method: + +```python +my_progressbar.setTotal( + total:int +) +``` +- `total` is the new total number of steps the progress bar should have. + +### Manual ETA Calculation +If no `etaCalc` function is provided when creating the instance, the estimated time of completion can be set manually using the `setEta()` method: + +```python +my_progressbar.setEta( + eta:int +) +``` + +### Display the Progressbar +The progress bar can be displayed using the `print()` method: + +```python +my_progressbar.print( + current:int, + eta:int +) +``` +- `current` is the current step the progress bar is at. +- `eta` is an optional overwrite of the estimated time of completion. + +If no `eta` is provided, the progress bar will use the `etaCalc` function to calculate the estimated time of completion. If no `etaCalc` function was provided when creating the instance, the progress bar will not display an estimated time of completion. + +### Build a Progressbar state +In case the progressbar should be built already but not displayed yet, the `buildSnapshot()` method can be used: + +```python +snapshot = my_progressbar.buildSnapshot( + current:int, + eta:int +) +``` +- `current` is the current step the progress bar is at. +- `eta` is an optional overwrite of the estimated time of completion. + +The method returns a string that can be printed to display the progress bar. + +:::tip[Example] +Let's build a snapshot of a progress bar with 100 steps and a current step of 50: + +```python +snapshot = my_progressbar.buildSnapshot( + 50 +) + +print(snapshot) +``` +Now we can update the progress bar by printing the snapshot again with a new current step: + +```python +snapshot = my_progressbar.buildSnapshot( + 75 +) + +print(snapshot, end="\r") +``` +We give the extra argument `end="\r"` to overwrite the current line in the console with the new progress bar. Otherwhise we end up with a new line for each progress bar update. + +This way we end up with a smooth progress bar that updates in place. +::: + +### Progressbar Eta Manager +The library also has a class called `ProgressbarEtaManager` that automatically estimates the time of completion for a progress bar. + +### Creating a Progressbar Eta Manager Instance +A new Progressbar Eta Manager instance can be created by calling the class with the following arguments: + +```python +my_eta_manager = ProgressbarEtaManager() +``` + +The Progress Bar Eta Manager works by saving the time taken by each step and calculating the average. This way, at each step we need to run the `now()` function to log another step. + +### Update the Progressbar Eta Manager +The `now()` method can be used to log another step and update the estimated time of completion: + +```python +my_eta_manager.now() +``` + +### Get the Average Step Time +The average time taken per step can be accessed using the `getAvgEta()` method: + +```python +avg_eta = my_eta_manager.getAvgEta() +``` + +To get the estimated time of completion for a specific number of steps, we can multiply the average step time by the number of steps: + +```python +eta = avg_eta * steps +``` +But the `Progressbar` class already does this for us, so we don't need to worry about it. + +### Get the logged Step times +The logged step times can be accessed using the `getDurations()` method: + +```python +durations = my_eta_manager.getDurations() +``` +This returns a list of all the step times that have been logged in seconds. + +:::tip[Example] + +## Combine Progressbar and Progressbar Eta Manager +To plug the `Progressbar` and `ProgressbarEtaManager` together, we can create new `Progressbar` and `ProgressbarEtaManager` instances: + +```python +# Create a new Progressbar instance +my_new_progressbar = Progressbar() + +# Set a total number of steps +my_progressbar.setTotal(100) + +# Create a new Progressbar Eta Manager instance +my_new_eta_manager = ProgressbarEtaManager() +``` + +Let's say we have a loop that runs 100 times and we want to display a progress bar with an estimated time of completion: + +```python +# code from above +... + +# Loop 100 times +for i in range(100): + # Log a new step + my_new_eta_manager.now() + + # simulate some work + time.sleep(math.random(0, 1, 0.1)) + + # Get the estimated time of completion + eta = my_new_eta_manager.getAvgEta() * (100 - i) + + # Print the progress bar + my_new_progressbar.print(i + 1, eta) +``` + +This way we can display a progress bar with an estimated time of completion that updates in place. + +The randomly generated delay between `0` and `1` seconds allows us to test the ETA calculation. + +::: + + + +