Options
All
  • Public
  • Public/Protected
  • All
Menu

Class SpotifyIdScraper

Spotify Album/Artist ID Scraper

Finds Spotify album/artist pair for an AlbumEntity. On a successful match, the IDs are saved into AlbumEntity.spotifyId and ArtistEntity.spotifyId.

Hierarchy

Index

Constructors

Properties

Methods

Constructors

constructor

Properties

Private album

album: AlbumEntity

artist entity used as the primary data source for album/artist ID scrape

Private albumRepository

albumRepository: Repository<AlbumEntity>

TypeORM repository handling all data flow in/out of album table

Private artist

artist: ArtistEntity

artist entity, derived from the album.artist relation

Private artistRepository

artistRepository: Repository<ArtistEntity>

TypeORM repository handling all data flow in/out of artist table

dataReadFromLocal

dataReadFromLocal: boolean

Scrapers always check for a local copy of the target resource (using Scraper.checkForLocalRecord) before executing a scrape from an external resource. If the resource was found (and therefore no external calls made), this is set to true.

description

description: string

A simple, human-readble description of what is being scraped. Used for logging.

results

results: ResultBatch

Contains all results generated by Scraper.scrape, including recursive calls.

scrapeSucceeded

scrapeSucceeded: boolean

Flag indicating a sucessful scrape, set to true after non-error-throwing call to Scraper.scrape.

Protected spotifyApi

spotifyApi: SpotifyApi

Protected spotifyResponse

spotifyResponse: AlbumSearchResponse

Spotify data populated by calls to SpotifyIdScraper.requestScrape

verbose

verbose: boolean

Used to override .env settings and force-log the output of a given scraper.

Methods

checkForLocalRecord

  • checkForLocalRecord(): Promise<boolean>

Private extractCorrectAlbumArtistPair

  • extractCorrectAlbumArtistPair(): void

  • Conditional method runner to set album/artist IDs

    First, looks for an exact album/artist match. If there is no exact match, looks for a substring match. For example, a search for Master of Puppets will fail, as Spotify only has the album Master of Puppets (Remastered). However, there are other instances where both exist, such as Hurry Up, We're Dreaming and Hurry Up, We're Dreaming (Commentary). This method prioritizes the perfect match, but will settle for the substring if there is no other option.

    Returns void

Protected extractInfo

  • extractInfo(): void

Private extractMatchingAlbumArtistPair

  • extractMatchingAlbumArtistPair(strict?: boolean): void

printInfo

  • printInfo(): void

printResult

  • printResult(): void

requestScrape

  • requestScrape(): Promise<void>

Protected saveToLocal

  • saveToLocal(): Promise<void>

scrape

  • scrape(forceScrape?: boolean): Promise<void>

  • Entry point for initiating an asset scrape. General scrape outline/method order:

    1. Scraper.checkForLocalRecord
    2. If local entity was found, update class props and return.
    3. Scraper.requestScrape
    4. Scraper.extractInfo
    5. Scraper.scrapeDependencies
    6. Scraper.saveToLocal
    7. Update class props and return
    remarks

    This method should be considered unsafe - there are several points where this can throw errors. This is intentional, and allows easier support for relational data scraping/storage. Scraped assets may have a mixture of required and non-required dependencies, the status of which should be kept in mind when implementing Scraper.scrapeDependencies. A subclass should catch and log errors from non-required scrapes. However, errors from a required scrape should remain uncaught, so the original call to a Scraper.scrape will error out before [[Scraper.save]] is called for incomplete data.

    Parameters

    • Default value forceScrape: boolean = false

      If set to true, scrapes the external resource regardless of any existing local records

    Returns Promise<void>

Protected scrapeDependencies

  • scrapeDependencies(): Promise<void>

Protected scrapeErrorHandler

  • scrapeErrorHandler(error: Error): Promise<void>

Static Private sanitize

  • sanitize(rawString: string): string

Static scrapeDependencyArr

  • scrapeDependencyArr<T>(scrapers: T[], forceScrape?: boolean): Promise<ScrapersWithResults<T>>

Generated using TypeDoc