Showing 497 open source projects for "web crawler source code"

View related business solutions
  • Iris Powered By Generali - Iris puts your customer in control of their identity. Icon
    Iris Powered By Generali - Iris puts your customer in control of their identity.

    Increase customer and employee retention by offering Onwatch identity protection today.

    Iris Identity Protection API sends identity monitoring and alerts data into your existing digital environment – an ideal solution for businesses that are looking to offer their customers identity protection services without having to build a new product or app from scratch.
    Learn More
  • Award-Winning Medical Office Software Designed for Your Specialty Icon
    Award-Winning Medical Office Software Designed for Your Specialty

    Succeed and scale your practice with cloud-based, data-backed, AI-powered healthcare software.

    RXNT is an ambulatory healthcare technology pioneer that empowers medical practices and healthcare organizations to succeed and scale through innovative, data-backed, AI-powered software.
    Learn More
  • 1
    WebMagic

    WebMagic

    A scalable web crawler framework for Java

    WebMagic is a scalable crawler framework. It covers the whole lifecycle of crawler, downloading, url management, content extraction and persistent. It can simplify the development of a specific crawler. WebMagic is a simple but scalable crawler framework. You can develop a crawler easily based on it. WebMagic has a simple core with high flexibility, a simple API for html extracting. It also provides annotation with POJO to customize a crawler, and no configuration is needed. Some other...
    Downloads: 7 This Week
    Last Update:
    See Project
  • 2
    Heritrix

    Heritrix

    Internet Archive's open-source, web-scale, web crawler project

    Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometimes spelled heretrix, or misspelled or missaid as heratrix/heritix/heretix/heratix) is an archaic word for heiress (woman who inherits). Since our crawler seeks to collect and preserve the digital artifacts of our culture for the benefit of future researchers and generations, this name seemed apt.
    Downloads: 8 This Week
    Last Update:
    See Project
  • 3
    QR Code generator library

    QR Code generator library

    High-quality QR Code generator library in Java, TypeScript/JavaScript

    This project aims to be the best, clearest library for generating QR Codes. My primary goals are flexible options and absolute correctness. The secondary goals are compact implementation size and good documentation comments. This work is an independent implementation based on reading the official ISO specification documents. I believe that my library has a more intuitive API and shorter code length than competing libraries out there. The library is designed first in Java and then ported to...
    Downloads: 17 This Week
    Last Update:
    See Project
  • 4
    The Apache Struts web framework

    The Apache Struts web framework

    Mirror of Apache Struts

    The Apache Struts web framework is a free open-source solution for creating Java web applications. Web applications differ from conventional websites in that web applications can create a dynamic response. Many websites deliver only static pages. A web application can interact with databases and business logic engines to customize a response. Web applications based on JavaServer Pages sometimes commingle database code, page design code, and control flow code. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • Field Sales+ for MS Dynamics 365 and Salesforce Icon
    Field Sales+ for MS Dynamics 365 and Salesforce

    Maximize your sales performance on the go.

    Bring Dynamics 365 and Salesforce wherever you go with Resco’s solution. With powerful offline features and reliable data syncing, your team can access CRM data on mobile devices anytime, anywhere. This saves time, cuts errors, and speeds up customer visits.
    Learn More
  • 5
    IntelliJ Community Edition

    IntelliJ Community Edition

    IntelliJ IDEA & IntelliJ Platform

    IntelliJ Community is the open source upstream of JetBrains’ IntelliJ IDEA, forming the core of a powerful, extensible, and intelligent development environment. It provides foundational features like a robust editor with code completion, syntax highlighting, refactoring tools, version control integrations, terminal, debugger, and plugin architecture. Since it’s open, community developers can contribute to language supports, UI tweaks, and platform enhancements.
    Downloads: 2,163 This Week
    Last Update:
    See Project
  • 6
    Odigos

    Odigos

    Distributed tracing without code changes

    ...Manage and configure collectors via a convenient web UI. Installing Odigos takes less than 5 minutes, and requires no code changes.
    Downloads: 36 This Week
    Last Update:
    See Project
  • 7
    Codename One

    Codename One

    Cross-platform framework for building truly native mobile apps

    An open-source mobile-first toolkit for building high-quality, cross-platform native apps for Android, iOS, Desktop & Web. Rapid cross-platform app development using Java or Kotlin with 100% code reuse. Apps are compiled down to native code for maximum performance and a smooth user experience. Write, debug, and test apps all inside your IDE (IntelliJ, Eclipse, VSCode or NetBeans) using the Codename One simulator.
    Downloads: 19 This Week
    Last Update:
    See Project
  • 8
    SikuliX

    SikuliX

    SikuliX version 2.0.0+ (2019+)

    ...It uses image recognition powered by OpenCV to identify GUI components and can act on them with mouse and keyboard actions. This is handy in cases when there is no easy access to a GUI's internals or the source code of the application or web page you want to act on.
    Downloads: 152 This Week
    Last Update:
    See Project
  • 9
    J2CL

    J2CL

    Java to Closure JavaScript transpiler

    J2CL is a lightweight transpiler developed by Google that converts Java source code into highly optimized JavaScript designed to work seamlessly with the Closure Compiler. It allows developers to write applications in Java while targeting web environments, enabling strong type safety and code reuse across platforms. Unlike monolithic frameworks, J2CL focuses purely on transpilation, leaving optimization, bundling, and runtime concerns to the broader toolchain, which provides flexibility in modern development workflows. ...
    Downloads: 2 This Week
    Last Update:
    See Project
  • Data management solutions for confident marketing Icon
    Data management solutions for confident marketing

    For companies wanting a complete Data Management solution that is native to Salesforce

    Verify, deduplicate, manipulate, and assign records automatically to keep your CRM data accurate, complete, and ready for business.
    Learn More
  • 10
    JeecgBoot

    JeecgBoot

    Low-code enterprise web development platform

    JeecgBoot is a low-code platform built on Spring Boot that accelerates enterprise application development with online forms, code generation, and a modern Vue-based frontend. It can generate CRUD screens, data dictionaries, and menu structures from database schemas, producing clean starter code that developers can extend. The platform integrates common enterprise features—RBAC permissions, data scopes, dictionary management, logging, and file/OSS integration—so teams start from a...
    Downloads: 4 This Week
    Last Update:
    See Project
  • 11
    Hutool

    Hutool

    A set of tools that keep Java sweet

    Hutool is a small but comprehensive Java tool class library. Through static method encapsulation, it reduces the learning cost of related APIs, improves work efficiency, makes Java as elegant as a functional language, and makes the Java language "sweet". The tools and methods in Hutool come from each user's meticulous attention to detail. It covers all aspects of the underlying code of Java development. It is not only a sharp tool to solve small problems in large-scale project development,...
    Downloads: 7 This Week
    Last Update:
    See Project
  • 12
    J2ObjC

    J2ObjC

    A Java to iOS Objective-C translation tool and runtime

    J2ObjC is an open-source command-line tool from Google that translates Java source code to Objective-C for the iOS (iPhone/iPad) platform. This tool enables Java source to be part of an iOS application's build, as no editing of the generated files is necessary. The goal is to write an app's non-UI code (such as application logic and data models) in Java, which is then shared by web apps (using GWT), Android apps, and iOS apps.
    Downloads: 3 This Week
    Last Update:
    See Project
  • 13
    AtlantaFX

    AtlantaFX

    Modern JavaFX CSS theme collection with additional controls

    Modern JavaFX CSS theme collection with additional controls.
    Downloads: 12 This Week
    Last Update:
    See Project
  • 14
    Testcontainers Java

    Testcontainers Java

    Testcontainers is a Java library that supports JUnit tests

    Testcontainers for Java is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container. Use a containerized instance of a MySQL, PostgreSQL or Oracle database to test your data access layer code for complete compatibility, but without requiring complex setup on developers' machines and safe in the knowledge that your tests will always start with a known DB state. Any other...
    Downloads: 8 This Week
    Last Update:
    See Project
  • 15
    Takes

    Takes

    True object-oriented Java web framework without NULLs

    Takes is a true object-oriented and immutable Java8 web development framework. Pay attention that UTF-8 encoding is set on the command line. The entire framework relies on your default Java encoding, which is not necessarily UTF-8 by default. To be sure, always set it on the command line with file.encoding Java argument. We decided not to hard-code "UTF-8" in our code mostly because this would be against the entire idea of Java localization, according to which a user always should have a...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 16
    Jenkins

    Jenkins

    Build great things at any scale

    Jenkins is the leading open-source automation server that allows you to build great things at any scale. Jenkins is built with Java and provides hundreds of plugins for building, deploying and automating virtually anything, allowing you to focus on more important things. Jenkins is often used for building projects, running tests, analyzing static code and deployment. Whatever is done repetitively, Jenkins can most likely execute and execute well, saving you time and optimizing your...
    Downloads: 15 This Week
    Last Update:
    See Project
  • 17
    Eclipse Che

    Eclipse Che

    Next-gen container development platform, workspace server & cloud IDE

    Eclipse Che is a Kubernetes-native IDE that makes Kubernetes development accessible for development teams. It places everything a developer could need into containers in Kube pods including dependencies, embedded containerized runtimes, a web IDE, and project code. With the Kubernetes application in your development environment and an in-browser IDE, you can code, build, test and run applications exactly as they run on production from any machine.
    Downloads: 1 This Week
    Last Update:
    See Project
  • 18
    WxJava

    WxJava

    WeChat develops Java SDK, supports back-end development

    WeChat develops Java SDK, supports back-end development including WeChat payment, open platform, small program, enterprise WeChat, official account, etc. For project cooperation and negotiation, please contact WeChat binary0000(search and add friends in WeChat, please indicate the intention of coming, if you need to discuss about SDK issues, please refer to the group discussion below, do not add this WeChat). Important reminder for beginners: This project is only an SDK development kit, and...
    Downloads: 6 This Week
    Last Update:
    See Project
  • 19
    HTTP Kit

    HTTP Kit

    Clojure HTTP server/client library with WebSocket support

    http-kit is a minimalist, event-driven, high-performance Clojure HTTP server/client library with WebSocket and asynchronous support. A simple, high-performance event-driven HTTP client+server for Clojure. HTTP Kit is an (almost) drop-in replacement for the standard Ring Jetty adapter. So you can use it with all your current libraries (e.g. Compojure) and middleware. Using an event-driven architecture like Nginx, HTTP-kit is very, very fast. It comfortably handles tens of thousands of...
    Downloads: 8 This Week
    Last Update:
    See Project
  • 20
    RStudio

    RStudio

    RStudio is an integrated development environment (IDE) for R

    RStudio is a powerful, full-featured integrated development environment (IDE) tailored primarily for the R programming language but increasingly supportive of other languages like Python and Julia. It brings together console, editor, plotting, workspace, history, and file-management panes into a unified interface, helping data scientists, statisticians, and analysts to work more productively. The IDE is cross-platform: there are desktop versions for Windows, macOS and Linux, as well as a...
    Downloads: 27 This Week
    Last Update:
    See Project
  • 21
    Serverless Java container

    Serverless Java container

    A Java wrapper to run Spring, Spring Boot, Jersey, and other apps

    The AWS Serverless Java Container library is a framework that allows developers to run existing or new Java web applications—built with frameworks such as Spring, Jersey, Spark, Struts—inside AWS Lambda with minimal modifications. It bridges the gap between traditional servlet or web-framework models and serverless functions by mapping HTTP events from API Gateway into requests your framework understands and routing responses back appropriately. This means you can keep much of your familiar...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 22
    Swagger Codegen

    Swagger Codegen

    Template-driven engine to generate documentation

    Swagger-Codegen contains a template-driven engine to generate documentation, API clients and server stubs in different languages by parsing your OpenAPI / Swagger definition. Simplify API development for users, teams, and enterprises with the Swagger open source and professional toolset. Find out how Swagger can help you design and document your APIs at scale. The power of Swagger tools starts with the OpenAPI Specification, the industry standard for RESTful API design. Individual tools to...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 23
    Sakura Boot

    Sakura Boot

    Framework on top of spring-boot that helps create back-ends

    This framework, built on top of Spring Boot, is designed to speed up the development of robust backends. It ensures adherence to best practices while allowing a high degree of customization. The framework is structured using a modular architecture, making it easy to add or remove features based on the project’s specific needs. While Spring Boot speeds up the development process, developers still require extensive knowledge, must ensure adherence to best practices, and often write repetitive...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 24
    Cerberus Testing

    Cerberus Testing

    The Open Source Test Automation Platform

    Cerberus Testing is a scalable test automation platform for teams searching to accelerate software delivery with confidence. Cerberus Testing is a test automation platform that allows iteration with quality and speed through test definition, execution, and reporting. The platform allows the creation of automated test cases through an easy-to-use web interface without any need for development skills - automated test implementation and management now become available for all elements of the...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 25
    AWS X-Ray Java Agent

    AWS X-Ray Java Agent

    The official AWS X-Ray Auto Instrumentation Agent for Java

    The AWS X-Ray Java Agent is a drop-in solution that enables the propagation of X-Ray traces within your Java web applications and servlet-based microservices. This includes automatic tracing for supported frameworks and libraries, including Spring, the AWS SDK, Apache HTTP clients, and JDBC-based SQL queries. The agent enables you to use the X-Ray SDK out of box, and requires no code changes to enable the basic propagation of traces. See the chart below for the current feature parity between...
    Downloads: 0 This Week
    Last Update:
    See Project
  • Previous
  • You're on page 1
  • 2
  • 3
  • 4
  • 5
  • Next
MongoDB Logo MongoDB