We just pushed out version 0.9, which is a big, big update to the system. This release includes several upgrades to our back-end architecture (allowing larger jobs), a Java API (allowing programmatic access), an easy-to-use job form (allowing easier access), and a bunch of other cool things!
Here’s a list of the specific features:
- Large crawls are now supported. Crawl up to 10 million pages per job!
- The API is officially released. Submit jobs, download results and much more using Java.
- A much easier-to-use job form. We realized the old job form was a bit clunky. The new one is much easier to understand.
- To go along with the new job form, we’ve updated the entire portal to be easier to navigate and use.
- You can now load in external JARs into your 80Apps. This lets developers use third-party code more easily.
- Several improvements to the crawler, including:
- Options to select your type of crawl. Choose among fast, comprehensive, and breadth-first.
- Crawler now crawls https:// pages.
- Crawler tries to fetch a page more than once before giving up.
Since we just released 0.9, I suppose that technically makes us 0.1 from a beta exit! Some of the upcoming features are:
- Finalizing the payment system in preparation for beta exit and charging actual money.
- Providing useful default 80Apps for all users (this is also in preparation for the app store model we’ll be pursuing).
See full release log details at http://80legs.pbworks.com/Release-Log.