• Home

  • Custom Ecommerce
  • Application Development
  • Database Consulting
  • Cloud Hosting
  • Systems Integration
  • Legacy Business Systems
  • Security & Compliance
  • GIS

  • Expertise

  • About Us
  • Our Team
  • Clients
  • Blog
  • Careers

  • VisionPort

  • Contact
  • Our Blog

    Ongoing observations by End Point Dev people

    Paginating API call with Radian6

    Marina Lohova

    By Marina Lohova
    August 24, 2012

    I wrote about Radian6 in my earlier blog post. Today I will review one more aspect of Radian6 API - call pagination.

    Most Radian6 requests return paginated data. This introduces extra complexity of making request several times in the loop in order to get all results. Here is one simple way to retrieve the paginated data from Radian6 using the powerful Ruby blocks.

    I will use the following URL to fetch data:

    /data/comparisondata/1338958800000/1341550800000/2777/8/9/6/

    Let’s decypher this.

    • 1338958800000 is start_date, 1341550800000 is end_date for document search. It’s June, 06, 2012 - July, 06, 2012 formatted with date.to_time.to_i * 1000.

    • 2777 is topic_id, a Radian6 term, denoting a set of search data for every customer.

    • 8 stands for Twitter media type. There are various media types in Radian6. They reflect where the data came from. media_types parameter can include a list of values for different media types separated by commas.

    • 9 and 6 are page and page_size respectively.

    First comes the method to fetch a single page.

    In the Radian6 wrapper class:

    def page(index, &block)
      data = block.call(index) 
      articles, count = data['article'], data['article_count'].to_i  
      [articles, count]
    end
    

    A data record in Radian6 is called an article. A call returns the ‘article’ field for a page of articles along other useful fields.

    Now we will retrieve all pages of data from Radian6:

    def paginated_call(&block)
      articles, index, count = [], 0, 0
      begin
        index += 1
        batch, count = page(index, &block)
        articles += batch 
      end while count > 0
      articles
    end
    

    Time to enjoy the method! I’m using httparty gem to make requests to API.

    paginated_call do |page|
      get("/data/comparisondata/1338958800000/1341550800000/2777/8/#{page}/1000/")
    end
    

    Thanks for flying!

    rails api


    Comments