• Home

  • Custom Ecommerce
  • Application Development
  • Database Consulting
  • Cloud Hosting
  • Systems Integration
  • Legacy Business Systems
  • Security & Compliance
  • GIS

  • Expertise

  • About Us
  • Our Team
  • Clients
  • Blog
  • Careers

  • VisionPort

  • Contact
  • Our Blog

    Ongoing observations by End Point Dev people

    CouchDB pagination with couchdb-python

    Miguel Alatorre

    By Miguel Alatorre
    July 2, 2013

    I’ve been working with couchdb-python v0.8 to meet some of a project’s CouchDB needs and ran into an unfortunate shortcoming.

    Retrieving database rows is as easy as:

    for row in db.view(mapping_function):
        print row.key

    However, all the rows will be loaded into memory. Now, for small databases this is not a problem but lucky me is dealing with a 2.5 million+ document database. Long story short: kaboom!

    There is an excellent blog post by Marcus Brinkmann that details the memory issue and also provides a pagination solution. However, couchdb-python v0.9 (released 2013-04-25) does provide its own solution: the iterview method. If you’ve been able to manage small databases with v0.8 but are anticipating larger and larger databases, be sure to upgrade!

    couchdb python