Skip to content

2015

  • in podcast
  • 4 min read

Install Hadoop on AWS Ubuntu Instance

Step 1: Create an Ubuntu 14.04 LTS instance on AWS. Welcome to Continuous Improvement, the podcast where we explore the world of personal and professional growth. I'm your host, Victor. In today's episode, we will delve into the intricate process of setting up a Hadoop cluster on an Ubuntu 14.04 LTS instance on AWS. If you've ever wanted to master the art of big data processing, this episode is for you.

Let's jump right in, shall we? Step 1, create an Ubuntu 14.04 LTS instance on AWS. Once you have that set up, we can move on to step 2: connecting to the instance. To do this, make sure you have the necessary key file, and then use the SSH command followed by the IP address of your instance. Easy peasy, right?

Step 3 involves installing Java, a key requirement for our Hadoop setup. We'll be using Oracle Java 6, so I'll walk you through the process of adding the repository, updating, and installing Java. Don't worry, I'll be sure to include all the necessary commands in the podcast description for your reference.

Now, let's move on to step 4: adding a Hadoop user. By creating a new group and user, we ensure proper management of the Hadoop environment. It's a crucial step in our journey towards a seamless Hadoop setup.

In step 5, we'll establish a password-free login by generating an SSH key. This will make it easier for remote access and interaction with your Hadoop cluster.

Once we've set up the connection, it's time to test it in step 6. You'll be able to verify the connection by using the SSH command again, this time connecting to "localhost." If everything goes smoothly, we can consider this step complete!

Moving forward to step 7, we'll download and install Hadoop itself. I'll guide you through the process of navigating to the correct directory, downloading the necessary files, extracting them, and making some minor adjustments like renaming folders and setting up ownership.

Step 8 is all about updating your .bashrc file. I'll explain this in more detail during the podcast, but essentially, we'll be adding some important environment variables for Hadoop and Java. This ensures that the necessary paths are set correctly for smooth operation.

In step 9, we'll dig deeper into Hadoop configuration. We'll be modifying the hadoop-env.sh file within the Hadoop configuration directory. This step is essential for ensuring that Hadoop is running on the correct version of Java, among other crucial settings.

Step 10 involves creating a temporary directory for Hadoop. This is where Hadoop will store its temporary data, so we want to make sure it's set up correctly with the proper permissions.

Moving along to step 11, we'll be adding configuration snippets. These are additional files that we'll need to modify to fine-tune Hadoop for our specific setup. I'll guide you through the process and explain the importance of each file.

In step 12, we'll format the HDFS (Hadoop Distributed File System). This step is crucial for preparing the Hadoop cluster for data storage and processing. I'll explain the ins and outs of this process, so don't worry if you're not too familiar with it.

Step 13 gets us closer to the finish line as we start Hadoop! Using the relevant command, we'll start all the necessary processes for our Hadoop cluster, so get ready to witness the power of big data in action.

Step 14 enables us to check if all the processes are up and running. By using the "jps" command, we can ensure that Hadoop is functioning as expected. It's always a good idea to double-check before proceeding further.

Ready for a quick breather? In step 15, we'll learn how to stop Hadoop. I'll walk you through the necessary command to gracefully shut down your Hadoop cluster, ensuring that all processes are stopped correctly.

Finally, in step 16, we'll learn how to start Hadoop again. This process is useful for restarting your cluster after making changes or simply resuming your big data endeavors. It's always good to have this knowledge at your disposal.

And there you have it! A comprehensive guide to setting up a Hadoop cluster on an Ubuntu 14.04 LTS instance on AWS. I hope you found this episode informative and useful for your own continuous improvement journey.

If you'd like to access the detailed commands and steps mentioned in this episode, please visit our podcast website or refer to the podcast description.

Thank you for joining me on this episode of Continuous Improvement. If you have any questions, suggestions, or topics you would like me to cover in future episodes, please reach out. Remember, learning is a lifelong journey, and with each step we take towards improvement, we grow and evolve.

Stay tuned for our next episode, where we'll explore another exciting subject. Until then, keep striving for greatness and never stop improving.

  • in zh
  • 2 min read

在AWS Ubuntu實例上安裝Hadoop

第一步： 在AWS上創建一個Ubuntu 14.04 LTS實例。

第二步： 連接到該實例。

chmod 400 yourKey.pem
ssh -i yourKey.pem ubuntu@your_instance_ip

第三步： 安裝Java。

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java6-installer
sudo update-java-alternatives -s java-6-oracle
sudo apt-get install oracle-java6-set-default

第四步： 添加一個Hadoop用戶。

sudo addgroup hadoop
sudo adduser --ingroup hadoop hduser

第五步： 為免密碼登錄創建一個SSH鍵。

su - hduser
ssh-keygen -t rsa -P ""
cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

第六步： 測試連接。

ssh localhost
exit

第七步： 下載並安裝Hadoop。

cd /usr/local
sudo wget [http://apache.01link.hk/hadoop/core/hadoop-1.2.1/hadoop-1.2.1.tar.gz](http://apache.01link.hk/hadoop/core/hadoop-1.2.1/hadoop-1.2.1.tar.gz)
sudo tar -xzvf hadoop-1.2.1.tar.gz
sudo mv hadoop-1.2.1 hadoop
chown -R hduser:hadoop hadoop
sudo rm hadoop-1.2.1.tar.gz

第八步： 更新.bashrc。

su - hduser
vim $HOME/.bashrc

# 在文件末尾添加以下內容：
export HADOOP_PREFIX=/usr/local/hadoop
export JAVA_HOME=/usr/lib/jvm/java-6-sun
unalias fs &> /dev/null
alias fs="hadoop fs"
unalias hls &> /dev/null
alias hls="fs -ls"
export PATH=$PATH:$HADOOP_PREFIX/bin

然後用:wq保存並執行.bashrc。

source ~/.bashrc

第九步： 配置Hadoop，以hduser身份登錄。

cd /usr/local/hadoop/conf
vim hadoop-env.sh

# 在文件中添加以下行：
export JAVA_HOME=/usr/lib/jvm/java-6-oracle
export HADOOP_CLASSPATH=/usr/local/hadoop

使用:wq保存並退出。

第十步： 為Hadoop創建一個臨時目錄。

exit
sudo mkdir -p /app/hadoop/tmp
sudo chown hduser:hadoop /app/hadoop/tmp
sudo chmod 750 /app/hadoop/tmp

第十一步： 添加配置段落。

su - hduser
cd /usr/local/hadoop/conf
vim core-site.xml

# 將以下內容放在<configuration> ... </configuration>標籤之間：

在這裡插入您的Hadoop配置。

# 使用 :wq保存並退出

如有需要，請繼續配置您的其他文件。

第十二步： 格式化HDFS。

/usr/local/hadoop/bin/hadoop namenode -format

第十三步： 啟動Hadoop。

/usr/local/hadoop/bin/start-all.sh

第十四步： 檢查所有進程是否正在運行。

jps

第十五步： 要停止Hadoop，輸入以下命令：

/usr/local/hadoop/bin/stop-all.sh

第十六步： 要再次啟動Hadoop。

/usr/local/hadoop/bin/start-all.sh

你現在已經準備好開始了！祝你玩得開心 :)

  • 1 min read

Install MongoDB on Mac OS X

First, install Homebrew, which is the missing package management tool for OS X:

> ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Next, update the formulae:

> brew update

Then, install MongoDB:

> brew install mongodb

To have launchd start MongoDB at login, run:

> ln -sfv /usr/local/opt/mongodb/*.plist ~/Library/LaunchAgents

To load MongoDB immediately, execute:

> launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mongodb.plist

Alternatively, if you don't want or need launchctl, you can simply run:

> mongod --config /usr/local/etc/mongod.conf

Also, create the data directory:

> sudo mkdir -p /data/db

Don't forget to change the permissions:

> sudo chown "$(whoami)" /data/db

Finally, start your database:

> mongod
  • in podcast
  • 2 min read

Install MongoDB on Mac OS X

First, install Homebrew, which is the missing package management tool for OS X:

Hello everyone and welcome to another episode of Continuous Improvement. I'm your host, Victor. In today's episode, we're going to talk about installing and setting up MongoDB on your Mac using Homebrew. If you're new to MongoDB or need to refresh your memory, you've come to the right place.

Before we dive in, make sure you have Homebrew installed on your system. If you don't, don't worry, I'll guide you through the process. Open up your terminal and type in the following command:

> ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Once Homebrew is installed, let's update the formulae by running:

> brew update

With Homebrew up to date, we're ready to install MongoDB. Type in the following command:

> brew install mongodb

Great! Now that MongoDB is installed, let's make sure it starts automatically on login. Run the following command:

> ln -sfv /usr/local/opt/mongodb/*.plist ~/Library/LaunchAgents

If you want MongoDB to load immediately, you can execute:

> launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mongodb.plist

Alternatively, if you prefer not to use launchctl, you can start MongoDB simply by running:

> mongod --config /usr/local/etc/mongod.conf

Remember to create the data directory by running the following command:

> sudo mkdir -p /data/db

To change the directory permissions, use the following command:

> sudo chown "$(whoami)" /data/db

Lastly, to start your MongoDB database, type in:

> mongod

And there you have it! MongoDB is now successfully installed and running on your Mac. Remember, if you encounter any difficulties during the installation process or have any questions, feel free to leave a comment or reach out to me.

That's it for today's episode of Continuous Improvement. I hope you found this guide useful and that you're now ready to make the most out of MongoDB on your Mac. Stay tuned for future episodes where we'll continue exploring different topics related to continuous improvement. Don't forget to subscribe to the podcast and leave a review if you enjoyed this episode. Until next time, happy coding!

[End of Episode]

  • in zh
  • 1 min read

在 Mac OS X 上安裝 MongoDB

首先，安裝 Homebrew，這是 OS X 缺少的套件管理工具：

> ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

接著，更新公式：

> brew update

然後，安裝 MongoDB：

> brew install mongodb

要讓 launchd 在登入時開始 MongoDB，執行：

> ln -sfv /usr/local/opt/mongodb/*.plist ~/Library/LaunchAgents

要立即加載 MongoDB，執行：

> launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mongodb.plist

或者，如果你不想或不需要 launchctl，你可以簡單地執行：

> mongod --config /usr/local/etc/mongod.conf

同時，創建數據目錄：

> sudo mkdir -p /data/db

不要忘記更改權限：

> sudo chown "$(whoami)" /data/db

最後，啟動您的數據庫：

> mongod
  • 1 min read

Enable Automatic Login for OS X El Capitan

The Problem

I just performed a fresh install of OS X El Capitan. I attempted to enable automatic login, but it remained locked. This was frustrating.

The Solution

  1. Go to System Preferences and select Security & Privacy. Turn off FileVault for the disk.
  2. Navigate to Users & Groups and click Change Password.
  3. Rather than using my iCloud password to log in and unlock this Mac, I opted to use a separate password.
  4. I set my new password.
  5. I clicked on Login Options and enabled automatic login for my account.

Yay, it worked!

  • in podcast
  • 2 min read

Enable Automatic Login for OS X El Capitan

The Problem

Welcome back, everyone, to another episode of Continuous Improvement. I'm your host, Victor, and today we're going to dive into a frustrating problem I recently encountered while performing a fresh install of OSX El Capitan. Stick around to hear how I managed to find a solution.

So, picture this: I had just completed the fresh install of OSX El Capitan on my computer, and I was eager to set up automatic login. But to my dismay, it just wouldn't work. It was locked, no matter what I tried. Frustration was an understatement at that point.

But don't worry, folks, because where there's a problem, there's always a solution. And lucky for you, I've got the step-by-step guide to help you resolve this annoying issue. Let's jump right into it.

Step one: Open up your System Preferences and navigate to the Security & Privacy section. Take a look around, and what you want to do here is turn off FileVault for the disk. This can often be the culprit behind automatic login being locked.

Step two: Now, it's time to head over to the Users & Groups section. Here, you'll find the option to change your password. Click on it and follow the prompts to set a new password. Remember, folks, this is an important step towards unlocking automatic login.

Step three: Here's where things get interesting. Instead of using your iCloud password to log in and unlock your Mac, choose to use a separate password. It may seem a bit counterintuitive, but trust me on this one.

Step four: Set your new password. Make sure it's strong and something you'll remember. It's always good to prioritize security, especially when dealing with automatic login.

Step five: Click on Login Options within the Users & Groups section. You'll find an option to enable automatic login for your account. Go ahead and enable it, and then restart your Mac just to be on the safe side.

And just like that, my friends, you've successfully unlocked automatic login on your freshly installed OSX El Capitan. Give yourself a pat on the back. It may have been a frustrating journey, but when you overcome a challenge, the feeling of accomplishment is simply unbeatable.

Well, that wraps up today's episode of Continuous Improvement. I hope you found this step-by-step guide helpful and that it saves you from the headache I experienced. Remember, folks, continuous improvement is all about learning from our struggles and finding solutions. Until next time, take care!

  • in zh
  • 1 min read

啟用 OS X El Capitan 的自動登錄

問題

我剛剛進行了 OS X El Capitan 的全新安裝。我試圖啟用自動登錄，但是它仍然被鎖定。這讓我感到困惑。

解決方案

  1. 前往系統偏好設定並選擇安全與隱私權。為磁盤關閉FileVault。
  2. 導航到使用者與群組並點擊更改密碼。
  3. 我選擇使用另一個密碼而不是使用我的 iCloud 密碼來登錄並解鎖這台Mac。
  4. 我設定了我的新密碼。
  5. 我點擊了登錄選項並為我的帳戶啟用自動登錄。

耶，它成功了！

  • 1 min read

Ember Inject Controller

The Problem I Encountered:

After updating my Ember project to version 1.13.5, I received the following warning in the browser console:

_DEPRECATION: Controller#needs is deprecated. Please use Ember.inject.controller() instead._

However, I couldn't find any documentation on how to implement the new syntax.

The Solution:

Though it's marked as a private method in the Ember documentation, you can view it by selecting the "private" checkbox.

There are two ways to use Ember.inject.controller(): with and without specifying a controller name.

_App.PostController = Ember.Controller.extend({ posts: Ember.inject.controller() });_

When you don't specify the name of the controller, Ember uses the property name to look it up, as in:

posts: Ember.inject.controller('posts').

You will only need to specify the controller name if the property name and the controller name are different.

_App.PostController = Ember.Controller.extend({ myPosts: Ember.inject.controller('posts') });_

This way, you can successfully update your project to comply with the new Ember guidelines.

  • in podcast
  • 2 min read

Ember Inject Controller

The Problem I Encountered: Welcome to Continuous Improvement, the podcast where we explore solutions to common problems faced by developers. I'm your host, Victor, and today we are diving into a recent problem I encountered while updating my Ember project.

So, after updating to version 1.13.5, I was greeted with a warning in the browser console telling me that Controller#needs is deprecated. The warning kindly suggested that I should use Ember.inject.controller() instead. But, I was left wondering, how exactly do I implement this new syntax?

Luckily, I did some digging and found a solution that I'll be sharing with you today. Firstly, let's address the Ember documentation - it marks Ember.inject.controller() as a private method. But fear not, you can still access it by selecting the "private" checkbox. It's always helpful to have access to these hidden gems, isn't it?

Now, let's dive into the implementation. There are actually two ways to use Ember.inject.controller(). The first method is without specifying a controller name. For example, you can define it like this:

Victor (cont.): App.PostController = Ember.Controller.extend({ posts: Ember.inject.controller() });

What happens here is that Ember uses the property name, in this case 'posts', to look up the controller. So, whenever you access 'this.posts' in your code, Ember will automatically fetch the 'posts' controller for you.

But what if the property name and controller name are different? Well, that's where the second method comes in. You can specify the controller name using Ember.inject.controller(). For example:

Victor (cont.): App.PostController = Ember.Controller.extend({ myPosts: Ember.inject.controller('posts') });

Here we have 'myPosts' as the property name and 'posts' as the controller name. Now, whenever you access 'this.myPosts', Ember will fetch the 'posts' controller for you.

And there you have it - a solution to the deprecation warning you may face when updating your Ember project. By using Ember.inject.controller(), you can ensure your code complies with the latest Ember guidelines.

That's it for today's episode of Continuous Improvement. I hope this solution helps you tackle any similar issues you may encounter in your own projects. Remember, continuous improvement is the key to becoming a better developer.

If you have any questions or topics you'd like us to explore in future episodes, feel free to reach out to us. Thank you for tuning in, and until next time, keep coding and keep improving.