Application Integration

Turning Stack Exchange API Tags Into Streaming Topics

We have been profiling the Stack Exchange API a lot recently because of the ease in which you can turn different APIs into streaming content feeds using Streamdata.io. The design of the Stack Exchange API makes it particularly easy to proxy and translate its variety of parameters into streaming subscription topics. It possesses a tag API which when you combine with the many different sites available which you can pull from the sites API, you get a nice mix of interesting content streams across many different sectors.

The first set of tags we wanted to showcase are for the popular social coding website Stack Overflow, which highlights the popularity of different programming languages developers are publishing questions and answers to on a daily basis.

[
  {
    "count": 1605161,
    "name": "javascript"
  },
  {
    "count": 1406893,
    "name": "java"
  },
  {
    "count": 1203098,
    "name": "c#"
  },
  {
    "count": 1191010,
    "name": "php"
  },
  {
    "count": 1096818,
    "name": "android"
  },
  {
    "count": 942994,
    "name": "python"
  },
  {
    "count": 908486,
    "name": "jquery"
  },
  {
    "count": 744873,
    "name": "html"
  },
  {
    "count": 567552,
    "name": "c++"
  },
  {
    "count": 560689,
    "name": "ios"
  },
  {
    "count": 532600,
    "name": "css"
  },
  {
    "count": 516264,
    "name": "mysql"
  },
  {
    "count": 438119,
    "name": "sql"
  },
  {
    "count": 332640,
    "name": "asp.net"
  },
  {
    "count": 291325,
    "name": "ruby-on-rails"
  },
  {
    "count": 283799,
    "name": "objective-c"
  },
  {
    "count": 276720,
    "name": "c"
  },
  {
    "count": 267498,
    "name": ".net"
  },
  {
    "count": 262407,
    "name": "arrays"
  },
  {
    "count": 251521,
    "name": "angularjs"
  },
  {
    "count": 235636,
    "name": "r"
  },
  {
    "count": 232538,
    "name": "json"
  },
  {
    "count": 230859,
    "name": "sql-server"
  },
  {
    "count": 223297,
    "name": "node.js"
  },
  {
    "count": 218977,
    "name": "iphone"
  },
  {
    "count": 194371,
    "name": "ruby"
  },
  {
    "count": 190868,
    "name": "swift"
  },
  {
    "count": 187455,
    "name": "regex"
  },
  {
    "count": 186265,
    "name": "ajax"
  },
  {
    "count": 171966,
    "name": "xml"
  }
]

These are just the top tags available on Stack Overflow with counts available for the number of questions available for each tag–showing the popularity of each topic. Showcasing what developers are interested in each day when it comes to getting the answers they need. To help augment the list of tags from Stack Overflow, we wanted to also a set of tags from the ProWebmasters forum, another site operated by Stack Exchange, but showing a different set of potential topics.

[
  {
    "count": 425,
    "name": "discussion"
  },
  {
    "count": 82,
    "name": "support"
  },
  {
    "count": 56,
    "name": "status-completed"
  },
  {
    "count": 51,
    "name": "tags"
  },
  {
    "count": 46,
    "name": "bug"
  },
  {
    "count": 45,
    "name": "feature-request"
  },
  {
    "count": 31,
    "name": "suitable-question"
  },
  {
    "count": 30,
    "name": "tag-synonyms"
  },
  {
    "count": 23,
    "name": "closed-questions"
  },
  {
    "count": 22,
    "name": "migration"
  },
  {
    "count": 21,
    "name": "moderators"
  },
  {
    "count": 19,
    "name": "election"
  },
  {
    "count": 18,
    "name": "status-bydesign"
  },
  {
    "count": 17,
    "name": "off-topic"
  },
  {
    "count": 17,
    "name": "editing"
  },
  {
    "count": 17,
    "name": "specific-question"
  },
  {
    "count": 15,
    "name": "questions"
  },
  {
    "count": 13,
    "name": "close-reasons"
  },
  {
    "count": 12,
    "name": "design"
  },
  {
    "count": 11,
    "name": "reputation"
  },
  {
    "count": 9,
    "name": "catch-all"
  },
  {
    "count": 8,
    "name": "community-ads"
  },
  {
    "count": 8,
    "name": "looking-for-a-script"
  },
  {
    "count": 8,
    "name": "flagging"
  },
  {
    "count": 7,
    "name": "retagging"
  },
  {
    "count": 7,
    "name": "badges"
  },
  {
    "count": 7,
    "name": "seo"
  },
  {
    "count": 7,
    "name": "spam"
  },
  {
    "count": 6,
    "name": "community-wiki"
  },
  {
    "count": 6,
    "name": "comments"
  }
]

We profiled 100 of the top sites available on Stack Exchange the other week, with these two sets of tags representing just a slice of what is available. Any of these tags can become a topical stream, just by proxying the Stack Exchange Questions API, and passing in the tag as a single parameter. Providing a rich set of content to be potentially streamed, mined, filtered, and made available in web, mobile, and machine learning applications.

Next we’d like to take the top 25 tags, across the top 100 Stack Exchange sites, and turn them all on a stream for a week, and see what interesting insights we can uncover. Understand which sites and topics are the most active on a daily basis, are there particular times and days which some sites are more active, and other interesting elements. Providing more of a map of the activity occurring via the network of Stack Exchange sites, across may different topics, opening up a window into what is happening that you can see across regular web APIs.

AI in Finance

**Original source: streamdata.io blog