Home/ Questions/Q 3869
Next
In Process

askthedev.com Latest Questions

Asked: 2024-09-24T18:44:21+05:30 In: , ,

What steps should I follow to set up Apache Flink on WSL2 running Ubuntu in Windows 11?

I’ve been trying to get Apache Flink up and running on my WSL2 setup with Ubuntu in Windows 11, and honestly, it feels like I’m navigating a maze without a map! I keep hitting roadblocks, and I figured it’s time to reach out to the community and see if anyone can help.

So, here’s the deal: I’ve got WSL2 installed and everything seems to be working fine, but I’m a bit lost when it comes to setting up Flink. I’ve read a bunch of tutorials, but they all seem to skip crucial steps or assume you already know more than I do. I’m starting from scratch here, so I really need something that covers the basics without diving into all the nitty-gritty details that just confuse me further.

First, I need to make sure my environment is set up correctly. What’s the best way to check if WSL2 is really functioning as it should? I don’t want to waste time setting up Flink only to discover that something’s off with WSL2.

Next, do I need to install Java separately, or is it bundled with Flink? I’ve heard that Java is a requirement, but I don’t want to get into dependency hell. Also, if I do need to install Java, what version should I go for? I’ve heard different things, and it’s making my head spin.

Once I’ve got Java sorted out, can someone walk me through the actual Flink installation? Like, what packages do I need to grab, and is there a specific directory where I should be placing everything? I’ve seen some tutorials that mention using Docker, but honestly, I’m a bit wary of getting into containerization right now—too many moving parts!

Lastly, setting up a simple “Hello World” type of job to ensure that everything’s working is something I’d love to see. If anyone has a quick example or knows where I can find one, that would be awesome!

I know there are a ton of you who’ve done this already, so if you could share your steps or any tips and tricks, it would make my life a whole lot easier. Thanks in advance for whatever help you can provide!

Share

    Leave an answer

    You must login to add an answer.

    Need An Account,

    2 Answers


    1. To ensure your WSL2 setup is functioning correctly, first, run the command wsl --list --verbose in your Windows terminal. This will list all your installed WSL distributions along with their states and versions. Look for “Ubuntu” to confirm it’s correctly installed and running. Additionally, you can check for Linux kernel updates by running uname -r within your WSL2 terminal, which should return a version compatible with WSL2. If everything checks out, you can move on to installing Java, which is indeed a prerequisite for Flink. Java is not bundled with Flink, so you’ll need to install it separately. The recommended version for most versions of Flink is Java 11, as newer versions can sometimes introduce compatibility issues. You can install it using the command sudo apt install openjdk-11-jdk.

      Once Java is installed, download Apache Flink from the official website using wget or curl. For example, you could run wget https://archive.apache.org/dist/flink/flink-x.x.x-bin-scala_x.x.tgz, replacing “x.x.x” with the version you want to install. After downloading, extract the files with tar -xzf flink-x.x.x-bin-scala_x.x.tgz and move the extracted folder to your desired directory, like /usr/local/bin. To verify everything is working, you can run Flink’s local cluster with ./bin/start-cluster.sh from the Flink directory. For a simple “Hello World” job, you can use the built-in examples included with Flink by running ./bin/flink run examples/streaming/WordCount.jar and checking the output in the Flink Web UI running on localhost:8081. Following these steps should get you up and running smoothly!




    2. Apache Flink Setup on WSL2

      Getting Apache Flink Running on WSL2

      Setting up Apache Flink on WSL2 can be a bit overwhelming, but let’s break it down step-by-step!

      Check if WSL2 is Working

      First things first, let’s ensure WSL2 is running smoothly. Open your WSL terminal (Ubuntu) and run:

      uname -r

      You should see a kernel version starting with 5.x.x. If you see that, you’re good to go!

      Do You Need Java?

      Yes, Flink requires Java! But don’t worry, Java isn’t bundled with Flink, so you’ll need to install it separately. The recommended version for Apache Flink is Java 8 or Java 11.

      You can install Java with the command:

      sudo apt update && sudo apt install openjdk-11-jdk

      Installing Flink

      Alright, now let’s get Flink installed. You can download Flink from the official website:

      Apache Flink Downloads

      Choose the latest stable release (make sure it’s the binary release). After downloading, you can extract it using:

      tar -xzf flink--bin-scala_

      Then, move into the Flink directory:

      cd flink-

      No need for Docker right now; we want to keep it simple!

      Running a Hello World Job

      Great, now you’re ready to run your first Flink job. You can start the Flink cluster by running:

      bin/start-cluster.sh

      To create a simple "Hello World" job, here's a quick example:

      cat << EOF > HelloWorld.java
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;

public class HelloWorld {
    public static void main(String[] args) throws Exception {
        final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
        DataSet<String> text = env.fromElements("Hello", "World");
        text.print();
    }
}
EOF
javac -cp 'lib/*' HelloWorld.java
flink run -c HelloWorld HelloWorld.jar

      Final Tip

      If you run into issues, check the logs located in the log directory inside the Flink folder. They'll give you hints on what might be wrong!

      And there you have it! Feel free to ask more questions if you get stuck. Good luck with your Flink journey!


    Related Questions