Posted on

Introducing: the diggrtoolbox

by Florian Rämisch and Peter Mühleder on behalf of the diggr project

The diggr project released its swiss-army-knife toolset as open source software this week. Implemented in Python, the diggrtoolbox provides various helpful functions for researchers doing data analysis in the game and game culture studies. Often required methods and tools are packaged to be easily included in your existing code.

Platform name standardization

The library includes methods to acquire the latest diggr platform mapping which provides a unique set of standardized platform spellings, as well as mappings for various popular video game databases. See also this blogpost. 

Game title matching algorithm

The library also contains an algorithm for matching video game titles. It returns a value between 0 and 1, based on the titles’ similarity. Numbers in game titles get detected and  evaluated separately.

Further contents

Overall, the diggrtoolbox library contains the following subpackages:

  • configgr: Provides a class which helps to keep a program wide configuration.
  • deepget: A small helper easing access to data in deeply nested dicts/list, by separating the definition of the route and actual call.
  • linking:  Contains the matching algorithm for video game titles.
  • platform_mapping: Methods to acquire the latest diggr platform mapping.
  • schemaload: Verify JSON data loaded into a program is consistent with a given JSON-schema.
  • standardize: String standardization tools, e.g. to get data ready for string matching etc. Helps removing most of the unwanted symbols, and characters complicating string comparison.
  • treeexplore: Methods to compare deeply nested dict and list data structures. Especially helpful for searching for specific strings/values in deeply nested dicts.
  • zipaccess:  Allows access to a JSON document in a ZIP-File.

Acquisition, Usage, Documentation, Examples

The diggrtoolbox is open source, the source code is to be found on github. After cloning or downloading it can be installed with Pip.

The full documentation is to be found in the repository itself or online at Read The Docs and contains more information on all packages, modules and methods. Furthermore, we made great effort to provide a set of useful examples  to demonstrate the possible applications of the library as well as integration in existing research environments.

Posted on

Platform Mapping

by Tracy Hoffmann, Peter Mühleder, Florian Rämisch on behalf of the diggr team

The diggr team has recently published a mapping, reference and grouping for platform strings of various video game databases. It is meant to ease mapping, matching and interlinking entities of various game databases.

Problem Statement

In various video game databases (community or institution-driven) different paradigms and ideas are followed for naming platforms. Especially digital re-releases of older games pose a huge problem, as they are sometimes assigned to their initial platform, various new platforms, or some kind of new hybrid indicating that it is a re-release. This makes research across databases, especially interlinking of entities, difficult and sometimes almost impossible.

Due to the fact that most of the existing databases have grown over a long time, some new platforms (e.g. digital Re-Releases) were probably hard to fit into existing data models. The database administrators made different decisions regarding how to cope with the new platforms. As every database has a different purpose and audience, some aspects were valued higher than others in each case. As a result, the coping strategies differ slightly from one database to another. We propose a first draft (Request for comments) to unify these compromises.

Solution

This is meant as a first step towards unification of platform identification. We provide a “master” platform list and a mapping for seven popular video game databases to this master list. It is not exhaustive.

Paradigms

Here are our paradigms

  • any platform has a standardized spelling
  • the standardized spelling is always the common long version of the platform name
  • if there is an equivalent entry in the GAMECIP Platform Vocabulary, link it

Shortcomings

  • At this time every specific arcade or pc platform is mapped to a more general term (e.g. Arcade – Namco System 246 -> Arcade ).

Covered Plattforms

Currently (as of May 2018) we cover seven popular video game databases: ESRB (USA), GameFAQs(USA/intl.), MediaArt (J), Mobygames(USA/intl.), OGDB (USA/intl.), PEGI (EU) and USK (D).

Outlook and Call for Collaboration

The master list we developed follows our use cases and applications. There probably is no “everything-fits-all” master list, which can be used for all possible use cases. Therefore, we strongly encourage you to put it to use and expand on it in your own contexts. This will help to identify shortcomings of our master list and allow us to develop it further together. Feel free to contact us about mistakes in the list or ideas to provide better access to those mappings: team@diggr.link

Access

The data can be accessed in two forms: Either by cloning the repository on github[1] and using the provided platform mapping files. Or by using the github pages, which are served from the dist subdirectory as a static REST API[2]. Using the static REST API version is preferable, as it always serves to up-to-date version, while using a cloned git repository can result in usage of an out-of-date version, e.g. if you forget to pull the latest updates before using it. In some data sources there are platform groups (like “Game Archives”) which include individual platforms. A mapping is located at platform_groups.tsv in the tabular_data folder. The file diggr_vocab.tsv contains the entire standarized vocabulary we used for internal mappings. It also contains links to the GAMECIP Platform Vocabulary.

License and Copyright

JSON-exports contain information about the author, license and copyright information, a version and a date element. We claim no copyright on the data, it therefore is released as Creative Commons CC0 1.0 Universal. The software, which generates the mapping files is free software under the GNU General Public License v3.

[1] //github.com/diggr/platform_mapping

[2] //diggr.github.io/platform_mapping