So the Turkish government blocked Github.
It’s been a while since my last post but I have lots of excuses. I have found a job and now I am working as a software engineer in Företagsplatsen, a small company with huge expertise and ambition. Our lovely office is located at Gamla Stan(I couldn’t imagine working at an office there before). We have a great team of developers and sales people.
I got my work permit after a long and troublesome process. The process was preventing me from visiting my girlfriend for many months. Truth to be told, I still feel angry when I think about the case officer and her behavior for my application. But with great relief, it is over.
I had the chance to visit my girlfriend during Christmas after I got my work permit. Seeing her after a long time was the happiest moment for me for 2015. When I arrived at the terminal station in Kadikoy, my heart was literally pounding. I was so happy with her during those days; when I look back, I feel nothing but regret leaving her in Istanbul(Unfortunately, it was the logical thing to do). But I had another chance to visit her during February for a week and we will be visiting Lisbon(my girlfriend’s favorite city and my previous home) in June. Even if we have some troubles in our relationship right now, she is the one that makes me feel incredibly happy.
Unfortunately, I have little time for my thesis work so it is now on hold for some time.
I plan to write more from now on.
As a part of my thesis, I’m assigned to integrate Java Batch Processing(JSR-352) into Hops. The problem I need to solve is actually simple : How can you create a JSR-352 job dynamically and then submit it?
Everything started in a cold Stockholm day, I was reading JSR-352 specification and giving my best to understand it. With every page and every sip of tea, my curiosity was growing bigger and bigger. Something was not right! I turned the last page, read it quickly and it was done. YAY, THE ONLY WAY I CAN SUBMIT A JOB IS TO READ FROM A FILE! YOU KIDDIN ME!
Well, I found a workaround. It requires a string containing the xml file’s contents. In other words, you can submit jobs if you have a string describing the job itself in proper xml. I give you the class definition with a static method which runs the job based on the input string. Beware that you should add proper dependencies (for example, using Maven) for Jbatch into your project in order to make it work.
This is a lengthy guide to use adam-shell in Apache Zeppelin.
Adam is a library for exploring and manipulating genomic data. It is quite popular among people who spend their life trying to create horse-sized ducks and then duck-sized horses. To help them achieve their goals, here is a short guide for using Adam library in Apache Zeppelin.
One of the main reasons of Adam’s popularity comes from the fact that it is actually developed as a custom API on top of Apache Spark. So with some distributed computing knowledge, it enables you to use your computing cluster to full extent.
What is Apache Zeppelin? I really don’t want to define Zeppelin because it is hard to define as a whole. In very simplistic terms, it is a web based notebooks for whatever interpreter you have. Luckily we have Spark interpreter since the beginning of the Zeppelin as an incubating project under Apache umbrella. So how can we use Adam library in Zeppelin?
If you know Spark to some extent, you try to put jar files of Adam libraries into appropriate locations then configure you Spark setup to load them on start-up. Then you start to hope that when Zeppelin starts and creates Spark environment for you, everything’s gonna be OK. However, you are not a computer scientist. You don’t care about Spark configuration gimmicks. What should you do?
The answer is actually simple. Zeppelin also comes with a dependency loader.
If you get an error message saying that dependencies should be loaded before running spark interpreter, then you need to go and restart spark interpeter and try again. If you get a valid result, then you are good to go. What the lines above does is to fetch Adam library from maven repositories and load it into your environment. I don’t think I need to explain how can you get the different versions of Adam library with the same method.
Whatever you do with adam-shell, you can now do it with Zeppelin and enjoy Adam-Zeppelin integration while creating duck-sized horses.
Go my friends, create the next horse-sized duck and name after me!
HR People of the planet Earth, please unify the job application systems! Come on!
Last few days, I’ve learnt to submit my profile in hundred different ways.