Apache Camel Tutorial

Updated:

Apache Camel Tutorial

Wanting to get started with the Apache Camel framework? Check out this free tutorial on this popular integration toolset for Java.

I’ve written this Apache Camel tutorial for complete beginners. It introduces the core Camel concepts to you, and shows you how to use Maven and Eclipse to build and run your first ever Camel project.

What you’ll learn

By the end of this tutorial, you will:

  • Understand what Apache Camel is, and what it can do
  • Create a basic Apache Camel project using Maven
  • See how Apache Camel can be used to automatically move files from one location to another
  • See how logging is used
  • Run your application and test it out

This tutorial is suitable for you if:

  • You want to learn Apache Camel
  • You want to know how to create a Camel project
  • You’re just getting started with JBoss Fuse and want to know how to build Camel routes
  • You’re just getting started with Talend ESB and want to know the underlying concepts so that you can understand Mediation routes
  • You’re a Java developer and you want to level-up your skills with a powerful framework

You should have a little bit of Java experience, and be comfortable with using Eclipse.


Camel concepts for beginners

What is Apache Camel?

So what is Apache Camel and what is it used for?

Apache Camel is an integration library for Java. It’s a set of Java APIs that help you integrate and process data between different computer systems.

In other words, Camel is like glue between different applications.

Camel comes with components that let you integrate lots of different applications — everything from web services, to reading and writing files on disk. You can even connect Camel with web apps like Twitter and Salesforce.

You can think of Camel as a plumbing toolkit for Java. Just like real plumbing pipes, Camel takes data from one point, and pipes it to another. Along the way, the data can be changed, transformed, or sent through other pipes.

Included in your plumbing toolkit is a range of adaptors that fit all sorts of different apps and software systems. With a full range of tools at your disposal, it’s then up to you how you choose to build the plumbing.

Where did the idea for Camel come from? Camel was largely inspired by the book Enterprise Integration Patterns, a sort of academic textbook for integrating software systems. The authors of the book (now considered a ‘classic’!) took dozens of common use cases and distilled them into reusable patterns, describing their use and, in some cases, providing suggested code for how to implement them. The Camel developers thought it would be a great idea to build a Java framework that represented the ideals of the book. And so Apache Camel borrows heavily from the book.

Some facts worth noting about Camel:

Camel_toolkit

  • It’s built in Java – this might seem an obvious point to make, but once you understand this you’ll see that you have the full power of Java at your disposal
  • The entire code is completely open sourcecheck it out at Github - nothing is hiding behind expensive closed-source products
  • It’s not just for web services – it’s a general integration framework. This means you might choose to use Camel to build something to do with web services, but you don’t have to.
  • It comes with a huge library of components – if you can think of a system you’d like to interact with, somebody has probably already written a component for it; everything from pushing files to an Amazon cloud, to sending a tweet.
  • It’s quite mature – Apache Camel has truly come of age and forms the foundation of some commercially-sold integration products, like Red Hat Fuse ESB and Talend ESB.

What is Camel used for?

Almost any time you need to move data from A to B, you can probably use Camel. Think of the following examples:

  • Picking up invoices from an FTP server and emailing them to you
  • Taking files from a folder and pushing them into Google Drive
  • Taking messages from a JMS queue and using them to invoke a web service
  • Making a web service that allows users to retrieve customer details from a database

These are just a few examples. With the wide range of components available in Camel, the sky’s the limit.

Thinking in Camel

How do you think like a Camel? Let’s look at the main concepts.

Visualising a Camel route
Visualising a Camel route

The diagram above shows some of the core concepts in Camel. In this section I’m going to look at each in turn.

Route

The basic concept in Camel is the route. Routes are objects which you configure in Camel, which move your data from A to B.

To use the plumbing example from earlier, a route is a pipe that moves data from one place to another. It moves data between things called endpoints.

You can create routes in Camel either using a Java syntax, or using an XML syntax. Here’s a very simple Java route:

// This is a complete Camel route definition!
from("file:home/customers/new") .to("file:home/customers/old");

Endpoints

In Camel, an endpoint represents any other external system to Camel. For an example, at the start of a route, Camel receives a message from an endpoint.

Then, the message might be processed in some way — perhaps by an EIP — before being sent to another destination endpoint.

Components

To allow Camel to connect to an endpoint, it comes with a library of components. A component is simply like a plug that allows you to connect to an external system (such as a file on disk, a mailbox, or an app like Dropbox, Twitter, etc). You can also think of it like an adaptor.

A huge library of components… Any time that you need to put data into or out of an application, you’ll probably find that a Camel component already exists to do the job for you. This means you don’t need to waste time writing your own code to read a file, or invoke a web service. You just find the component you need, and use it.

Camel components are reusable, open source, and you can even contribute your own. Here are some of the most common components, and how you might reference them in an endpoint:

Component Purpose Endpoint URI
HTTP for creating or consuming web sites http:
File for reading and writing files file:
JMS for reading and writing to messaging queues jms:
Direct for joining your Camel routes together direct:

You can find a list of all the Camel components here.

As you can see, each component can usually read and write:

  • A component that is configured to write something is called a producer — for example, writing to a file on disk, or writing to a message queue.
  • A component that is configured to read something is called a consumer — for example, reading a file from disk, or receiving a REST request.

Between each endpoint, the data can also be transformed or modified, either by passing the data through another endpoint, or by using an EIP.

Enterprise Integration Patterns (EIPs)

EIPs are another important part of Camel. They do special processing on messages according to the patterns defined in the book, Enterprise Integration Patterns, that I mentioned earlier.

When you want to perform some common activities on a message, such as transformation, splitting and logging, you’ll use an EIP. Here are some common EIPs in Camel:

EIP name What it does Java syntax
Splitter Splits a message into multiple parts .split()
Aggregator Combines several messages into one message .aggregate()
Log Writes a simple log message .log()
Marshal Converts an object into a text or binary format .marshal()
From* Receives a message from an endpoint .from()
To* Sends a message to an endpoint .to()

(Yes, from and to are EIPs too!)

Camel Context

Finally, to run and manage your routes, Camel has a container called the Camel Context. Your routes run inside this engine. You could think of it almost like a mini application server.

When Camel starts, it reads your route definitions (in Java or XML), creates the routes, adds them to a Camel Context, and starts the Camel Context.

When Camel terminates, it shuts down your routes, and closes the Camel Context.

What does a Route look like?

So. Now you know that when you develop in Camel, you create routes that move data between endpoints, using components.

It’s probably easiest to understand all of this by looking at some code, right?

Although Camel is a library for Java, it can be configured using one of two languages - either Java or XML. In Camel-speak, these are known as DSLs (Domain Specific Languages).

Each route starts with a from, configured with a URI, that defines the endpoint that the data is coming from.

A route can consist of multiple steps — such as transforming the data, or logging it. But a route usually ends with a to instruction, which describes where the data will be delivered to.

A really simple route in Camel’s Java DSL could look something like this:

from("file:home/customers/new")
    .to("file:home/customers/old");

In this example, we use the File component (identified by the file: prefix) to move all incoming files in the customers/new folder, to the customers/old folder.

That same route above could be expressed in Camel’s XML DSL like this:

<route>
    <from uri="file:home/customers/new"/>
    <to uri="file:home/customers/old"/>
</route>

But… what if we wanted to add another step in our route? Let’s say we want to log a message when we’ve received a file. Then we simply need to add our new step in between the existing steps. Like this:

from("file:home/customers/new")
    .log("Received a new customer!")
    .to("file:home/customers/old");

In the code above, a message is moved from the new folder to the old folder. In the middle, we use the Enterprise Integration Pattern (EIP) called log, which writes a simple Log message to the console.

And in the XML DSL, it would look something like this:

<route>
    <from uri="file:home/customers/new"/>
    <log message="Received a new customer!"/>
    <to uri="file:home/customers/old"/>
</route>

Now you know what a route looks like, let’s quickly look at how data flows through your routes.

What does data look like in Camel?

A nice envelope

What does this mean? It means that Camel treats data as individual messages – like letters flowing through a post office.

Each message is an individual unit. A message can be huge, or it can be very small. Camel has an object to represent messages, and helpfully it’s called Message.

A Message has a body, where the message content lives. It also has headers, which can be used to hold values associated with the message. The Message object is then passed along a route.

A Message is part of a Camel object called an Exchange. You’ll often see the term Exchange mentioned in Camel documentation. An Exchange is simply a message or interaction currently taking place inside your Camel route.

The important thing to understand about Camel’s message model is that the message body can contain almost any kind of Java object – it doesn’t have to be XML, JSON or even plain text for that matter.

The real power of Camel becomes clear when you start using different types of objects. Camel has a lot of built-in support for converting between different object types.

In fact, for many common file types, you might barely even have to write any code. (Less time writing boilerplate code? Sounds good, doesn’t it?)

You will learn more about Camel’s message model as you get more experienced with Camel!


Your first Camel project

In this part of the tutorial, I’m going to show you how to create a new Apache Camel project in Eclipse.

If you’ve never worked with Camel before, then by the end of this chapter, you’ll have created your first project, run it, and begun to understand the power of Camel.

We’re going to do this using a Maven archetype. Maven archetypes are like templates for new Java projects. Camel provides quite a few of them, to make it easier for you to start new projects.

For this tutorial, you will need:

Note for Mac users: If you’re using a Mac, I recommend using the Homebrew package manager to help you install Maven. Follow the instructions at http://brew.sh to install. Then, once Homebrew is installed, install Maven by typing brew install maven from a Terminal window.

Creating a Camel project in Eclipse

  1. On your computer, launch Eclipse.

  2. From the File menu, choose New -> Maven Project.

    New Maven Project in Eclipse

  3. On the first step of the New Maven Project wizard, make sure that Create a simple project remains unchecked. Then choose Next to accept the defaults.

    Maven Wizard - first step

  4. The next step asks you to select a Maven archetype for your project, from a list of available archetypes. (You might need to wait a while for the full list to load.)

    Type camel in the filter box to search for just Camel archetypes. Then choose the archetype in the group org.apache.camel.archetypes group, with the artifact name camel-archetype-java.

    Maven Wizard - choosing an archetype

    Then click Next.

  5. Finally, on the last step of the new Maven project wizard, give your new application’s groupId and artifactId:

    Maven Wizard - choosing artifact ID and group

    Once you’ve done that, click Finish.

  6. Now your Camel project is created in Eclipse!

    In a moment, we’ll run the app and see what it does.

    But first, let’s take a look at the structure of a typical Camel project.

Looking inside the project

A Camel project created from the camel-archetype-java archetype will have a folder structure that looks like this:

project/
└ src/
  ├ main/
  │ ├ java/
  │ │  └ com/
  │ │  └ example/
  │ │    ├ MainApp.java
  │ │    └ MyRouteBuilder.java
  │ └ resources/
  │    └ log4j2.properties
  └ test/
    ├ java/
    └ resources/

The main code resides in MyRouteBuilder.java. This class extends Camel’s RouteBuilder class, and this is where your routes should go:

package com.cleverbuilder;

import org.apache.camel.builder.RouteBuilder;

public class MyRouteBuilder extends RouteBuilder {

    public void configure() {
      // Your route definitions go here!
    }
}

Here, you add route definitions to the configure() method that describe each route you want Camel to create.

If you look at the code, you’ll see that the MyRouteBuilder class in the Eclipse project comes with a route defined inside it already.

For your convenience, the project also includes another Java class, MainApp, which is used to run your route. It uses the familiar Java public static void main() method to initialise MyRouteBuilder, which creates the routes:

public static void main(String... args) throws Exception {
    Main main = new Main();
    main.addRouteBuilder(new MyRouteBuilder());
    main.run(args);
}

Now let’s look at the code and see what this project does.

What does the project do?

The demo project defines a route using the code below. The code is given in a special Camel syntax, or DSL (Domain-specific Language):

from("file:src/data?noop=true")
  .choice()
    .when(xpath("/person/city = 'London'"))
      .log("UK message")
      .to("file:target/messages/uk")
    .otherwise()
      .log("Other message")
      .to("file:target/messages/others");

This code looks complicated, but it really just moves files from one folder to another.

It receives files from source folder using from(). The output folder is determined based on the content of the message. (This is Camel’s content-based routing function in action!)

Let’s look at each part of this statement in detail:

  • The choice() EIP states that we’re starting a content-based routing section. In other words, we are configuring Camel to process any incoming message according to a condition.

  • The when() keyword uses XPath (the language for navigating XML documents) to test whether the <city> element in the XML message equals “London”. If so, it writes a log message, "UK message", and then moves the file to the target/messages/uk folder.

  • The otherwise() keyword defines what should happen if the previous test returns false. If it’s false, it writes a log message, "Other message", and moves the file to the target/messages/others folder.

You’ll see that, although it’s been formatted for easier reading, the entire route is a single Java statement – it starts with the method call from(), and ends with a to() method call and the statement terminator, ;.

To get you started, the demo route provides you a couple of demo data files to test with. These are in the folder src/data. The two files are called message1.xml and message2.xml.

When the route runs, these two files will be picked up by Camel, inspected to see whether they are London files, and routed accordingly.

Now let’s run the route and see what happens.

Running the Camel application

To run the route:

  1. In the project tree, find the class MainApp.java.

    Eclipse - find MainApp class

  2. Right-click the file and select Run As -> Java Application.

    Eclipse - run as Java application

    TIP: If you’re prompted by a Select Java Application dialog box, tell Eclipse to run the app as a Camel route by selecting Main – org.apache.camel from the list.

  3. Camel will launch, process the files, and you’ll see the output from the route in the Console window.

    Eclipse - Viewing the Console output

So….what just happened?

Camel found your route definitions (in MyRouteBuilder) and started a CamelContext, containing your single route.

Then, it started the route, which started the consumption of messages (from the src/data folder). Finally, it processed each file according to its content.

If you check the output folder (target/messages) now, you’ll see the two files have been sorted into their output folders:

Showing files sorted into their folders

The other thing to do is have a look at the console or log output. This shows exactly what happened when Camel processed the files:

  • The first file, message1.xml, had a city value of London, so it was routed to the uk folder and a log message: UK message was written to the console.

  • The second file, message2.xml, had a city value of Tampa, so it was moved to the others folder, and a log message: Other message was written.

So Camel has just processed some files according to some rules, and performed different actions on each file. And all without having to drop to low-level Java code.

This is a very basic demo of Camel, but you can start to see what it’s capable of. Using components and EIPs you can build almost any kind of data flow that you can think of.

What next?

You’ve finished with this tutorial! Now check out my other articles to help you learn Camel:

And now, it’s over to you. What do you want to build in Camel?

Questions about this tutorial? Feedback? Share your thoughts and your Camel plans in the comments below!

Leave a Comment

You can use Markdown in your comment. To write code, indent lines by 4 spaces.