Introducing Nebo's Golbot: A Sports Statistics API Integration for Slack

The Problem

Let’s assume you’re interested in a few different sports: the NFL, NHL, and English Premier League.

On Monday morning, you may want to check all the scores from the weekend. One option would be to go to a sports-related website (ESPN, for example) and browse the respective sections for each of those leagues.

Before you know it, you’ve clicked dozens of times, absorbed tons of information you weren’t really looking for, and, hopefully, have found all the relevant scores from last weekend.

Of course, you could create an account, and, most likely, I assume, services like ESPN would allow you to reorganize your homepage/dashboard in a way that shows you the sports you’re interested in.

There has to be a better way.

The Basics

The majority of the conversation that takes place at our company takes place on Slack. What if I could get all the scores from the past weekend on Slack?

[Golbot enters stage left]

Each league is given a three or four-letter code. By issuing the “leagues” command, you can retrieve a list of the currently tracked leagues.

Retrieving Matches

Even though sports matches follow a fairly general structure, with a home and away team and home and away scores, each league stores this information in a different location and format. On top of that, I’ve discovered that sports-related APIs are amongst the most protected on the Internet.

I first attempted to find a centralized API that would source all the data I would need for this project, but I was unable to find an affordable solution. It turns out that Sports APIs are in extremely high demand and, as such, providers of such data require that users compensate them.

The environment around sports-related APIs is quite different from what a developer may be used to. Because of these difficulties, I decided each league would need to be individually integrated into Golbot from a different (free) source.

          
def self.get_nfl_matches(day = Time.now.in_time_zone.strftime("%d"), month = Time.now.in_time_zone.strftime("%m"), year = Time.now.in_time_zone.to_date.year.to_i)
  matches_created = []
  url = "http://www.nfl.com/schedules"
  uri = URI.parse(url)

  response = Net::HTTP.get_response(uri)

  if response.code == "200"
    doc = Nokogiri::HTML.parse(response.body)

    games = doc.css('ul.schedules-table').last
    games = games.css('li')

The method that retrieves the NFL schedule

          
def self.get_nhl_matches_by_day(day = Time.now.in_time_zone.strftime("%d"), month = Time.now.in_time_zone.strftime("%m"), year = Time.now.in_time_zone.to_date.year.to_i)
  matches_created = []
  games_url = "https://statsapi.web.nhl.com/api/v1/schedule?startDate=#{year}-#{month}-#{day}&endDate=#{year}-#{month}-#{day}&expand=schedule.boxscore"

  uri = URI.parse(games_url)

  response = Net::HTTP.get_response(uri)

  if response.code == "200"
    dates = JSON.parse(response.body)['dates']
    unless dates.blank?
      dates.each do |date_object|
        games = date_object['games']
        games.each do |game|

The method that retrieves the NHL schedule

          
def self.get_epl_matches_by_day(day = Time.now.in_time_zone.strftime("%d"), month = Time.now.in_time_zone.strftime("%m"), year = Time.now.in_time_zone.to_date.year.to_i)
  matches_created = []

  games_url = "http://www.espnfc.us/english-premier-league/23/scores?date=#{year}#{month}#{day}"
  uri = URI.parse(games_url)

  response = Net::HTTP.get_response(uri)

  if response.code == "200"
    doc = Nokogiri::HTML.parse(response.body)

    matches = doc.css('.score-box')
    unless matches.blank?
      matches.each do |game|

The method that retrieves the EPL schedule

As you can see in the screenshots above, the method that retrieves NFL matches retrieves that data from a different source than the equivalent method for retrieving NHL or EPL matches (the official NFL website, official NHL website, and ESPNFC, respectively).

Parsing the Result

While the NHL, NBA (shown below), and MLB all provide public-facing, free JSON feeds of schedules and scores, this type of feed is not available for other leagues, such as the NFL or MLS.

In cases where a JSON feed is not available, we can load the public-facing website into Nokogiri and parse the HTML markup itself. It’s not pretty, and changes in the markup will result in the code failing. Any markup changes will need to be reflected in the CSS selectors being used to parse the result.

Handling Time Zones

Another thing to note is that, because we’re acquiring statistics from multiple time zones, time formats will vary from match to match, and league to league. The English Premier League, for example, lists times in BST (UTC+1), while UEFA converts all times to the local time of the user.

When parsing the result from Nokogiri, we need to be sure to convert all times to a standard time zone. I’ve used “Eastern Time (U.S. & Canada)” for Golbot, which is defined in the Ruby on Rails configuration.

The "Today" Command

The "Find" Command

The “Find” command searches all matches for a specified team name. This can be useful if you’re unsure which teams in a specific city are being tracked, or if you would like a schedule of upcoming games for a specified team or city.

Trackers

While having league standings and game results at the tip of our fingers is fantastic, it would be even better if we could get live updates of games in progress:

Introducing the Golbot::Tracker.

Trackers can be created through the “Track” command, which accepts a specific format of “<team1> vs <team2>.”. If Golbot finds multiple matches that match your search query, it will return specific IDs for each match. You can then use the “Track-id” command to select the match you would like to track.

When the specified match begins, Golbot will begin a live feed, in the channel from which you originally asked it to track the match.

Twitter Integration

As most teams have an active social media presence, Golbot also tracks teams’ live Twitter feeds:

          
def self.twitter_update(league, team, since_tweet_id = nil)
  teams = {
    "ucl" => {
      "Atletico Madrid" => "atletienglish",
      "Leicester City" => "LCFC",
      "Real Madrid" => "realmadriden",
      "Bayern Munich" => "FCBayern",
      "Barcelona" => "FCBarcelona",
      "Juventus" => "juventusfcen",
      "AS Monaco" => "AS_Monaco_EN",
      "Borussia Dortmund" => "BVB",
      "Paris Saint-Germain" => "PSG_English",
      "Celtic" => "celticfc"
    },
    "mls" => {
      "Columbus Crew" => "ColumbusCrewSC",
      "Orlando City SC" => "OrlandoCitySC",
      "Chicago Fire" => "ChicagoFire",
      "New York City FC" => "NYCFC",
      "NY Red Bulls" => "NewYorkRedBulls",
      "Atlanta United FC" => "ATLUTD",
      "New England Revolution" => "NERevolution",
      "D.C. United" => "dcunited",
      "Toronto FC" => "torontofc",
      "Montreal Impact" => "impactmontreal",
      "Philadelphia Union" => "PhilaUnion",
      "Portland Timbers" => "TimbersFC",
      "Sporting Kansas City" => "SportingKC",
      "FC Dallas" => "FCDallas",
      "Houston Dynamo" => "HoustonDynamo",
      "San Jose Earthquakes" => "SJEarthquakes",
      "Real Salt Lake" => "RealSaltLake",
      "Vancouver Whitecaps FC" => "WhitecapsFC",
      "LA Galaxy" => "LAGalaxy",
      "Seattle Sounders" => "SoundersFC",
      "Minnesota United FC" => "MNUFC",
      "Colorado Rapids" => "ColoradoRapids"
    },
    "nba" => {
      "Washington Wizards" => "WashWizards",
      "Atlanta Hawks" => "ATLHawks",
      "Miami Heat" => "MiamiHEAT",
      ...

Golbot keeps track of the ID of the most recent tweet for any given Tracker object, ensuring it doesn’t repeat itself, repeat itself, repeat itself.

Nebo TV Dashboard Integration

One of my favorite things about Golbot is it’s integration with our TV Dashboard at the front of the office. Shout-out to our Creative Director, Pete Lawton, for designing the icons. The TV Dashboard communicates with Golbot in order to display upcoming matches for any Atlanta-based professional sports team.

This result is achieved through an API endpoint that uses the same methodology as the “Find” command, querying “Atlanta” as the search phrase, and limited by three days into the past or future. The first result is then displayed on the TV. If there are no upcoming matches, the most recent match would display.

          
def dashboard
  matches = []
  date = Time.now.in_time_zone.to_date

  games = ::Golbot::Match
    .where("date > ? AND date < ?", date.at_beginning_of_day, date.at_end_of_day + 3.days)
    .where("(((home ILIKE ?) OR (away ILIKE ?)) OR ((home ILIKE ?) OR (away ILIKE ?)))", '%atlanta%', '%atlanta%', '%falcons%', '%falcons%')
    .order(date: :asc)

  unless games.blank?
    games.each do |game|
      if game.date.present?
        matches << ["#{game.league.upcase}", "#{game.home} vs #{game.away}", "#{game.date.in_time_zone.strftime('%a, %b %-d, %-l:%M %P')}"]
      else
        matches << ["#{game.league.upcase}", "#{game.home} vs #{game.away}", "Time Unavailable"]
      end
    end
  end

  respond_with matches.as_json
end

Lesson Learned

While this project was entirely built on a whim, I did take away at least one very valuable lesson.

Beware of dependencies. Especially dependencies on someone else’s markup.

Relying on a third-party source for code that is integral to your application will result in the failure of your application, should that code no longer function properly.

This concern is well documented in the development world. Developers should already be aware for things like external JavaScript libraries or Ruby Gems, for example, but Golbot also relies on the format of a third-party’s HTML markup.

Overall, this causes a bit of unnecessary regular maintenance, where the process could be seamlessly automated. In retrospect, I would have made more effort to avoid this dependency, and probably spent more time trying to find and integrate an appropriate API.

Golbot: A Sports Statistics Slack Integration