Friday, August 22, 2014

HPC sessions at VMworld US 2014

Considering the number of sessions up for vote:
http://virtual-hiking.blogspot.com/2014/05/virtualized-hpc-and-customer-sessions.html

there were considerably fewer accepted sessions but still a few to pay attention to next week, either in your schedule or to review after the conference is over.  This is a short-list since it's last minute and my apologies if I'm missing a session so please let me know in the comments or twitter.  As a rule, I would check the master schedule onsite as times and dates may change.  In addition, during TAM day lunch on Sunday, there will be a table with Josh Simons, resident master of HPC in the Office of the CTO, and Matt Herreras, SLED SE manager, to answer informal questions about virtualizing HPC.

Virtualized-HPC-as-a-Service #vBrownBag Talk
Monday at 12:30pm in the Community hangspace:
http://professionalvmware.com/2014/08/vbrownbag-tech-talk-schedulevmworld-usa-2014/

INF1466: High-Performance Computing in the Virtualized Datacenter
This customer session was already full for Tuesday, 2-3pm.  It would be great to get a repeat session but no guarantees.  Edmond had in-depth results to share last year and I am looking forward to this one for more real-world experience.
 

VAPP1856: How to Engage with Your Engineering, Science, and Research Groups About Virtualization and Cloud Computing
This is with Josh Simons and Matt Herreras, Thursday 10:30-11:30am.

TEX1808: Data Plane Performance for NFV with VMware vSphere and Intel DPDK
This is with Bhavesh Davda from VMware's Office of the CTO and Edwin Verplanke, a Systems Architect for Intel, to discuss the latest on low latency networking.  A bit early on Wednesday at 8-9am but definitely still recommended.

VAPP1428: Hadoop as a Service: Utilizing VMware vCloud Automation Center and Big Data Extensions at Adobe
Discussing the business use-case but also performance recommendations and handling operations for high-utilization VMs, Monday 1-2pm and Wednesday 2:30-3:30pm.

Finally, the Office of the CTO and HPC team will have staffing at the VMware OCTO booth for additional questions or comments.

Monday, August 18, 2014

My VMworld 2014 Schedule

Hopefully everyone who is going to be attending VMworld in San Francisco this year has their accommodations already worked out.  The following is tentatively where I'll be presenting or outside of private customers meetings in a week.  I look forward to seeing you there!

Sunday - 8/24/14

I'll hopefully be working at the VCDX bootcamp at the local office in San Francisco.  I really enjoy working with the up and coming VMware architects in the industry.  I believe becoming a VCDX, and to be more specific, going through the preparation and panel process which includes focused analysis of design, tradeoffs and realities, really helped me gain a depth of understanding in architecture.  To be authoritative in one area is a first step, for example storage or networking or a specific application.  Then to become authoritative in multiple areas and orchestrate a solution is another step.  And in the VCDX process, to not only author solutions in front of customers but to do that in front of a prepared panel of experts and peer is another step altogether.  It would never be satisfying for me to sit across the table from potential architects and see them fail so it is my hope to be able to mentor and offer what insights I have as well as learn from their own experiences.  What is knowledge without perspective!

Monday - 8/25/14

12:30pm-12:45pm - #vBrownBag vHPC-as-a-Service: I'll be presenting in the Community Hangspace.  Basically a condensed lightning talk about the what/why/how of doing virtualized HPC-aaS.  If there are additional questions, that's great, as I'll be at the VMware Office of the CTO booth as well as around the venue for the next few days.

1:00-2:00pm - VAPP1428 Hadoop-as-a-Service: Chris Mutchler from Adobe and I will be giving this update on the most recent developments and examples around the growing possibilities for companies to evolve their data pipelines in an elastic, self-service, and scalable manner.

4:00-5:00pm - VAPP1807 Best Practices of Virtualizing Hadoop on vSphere - Customer Panel: Wouldn't miss this for for insight and perspective on other customers' high performance data analytics environments.

Tuesday - 8/26/14

1:00-6:00pm - Office of the CTO booth: On the Solutions Exchange floor at the VMware booth to answer questions and show off some of the bleeding edge work being done by VMware's OCTO organization.

Wednesday - 8/27/14

2:30-3:30pm - VAPP1428 Hadoop-as-a-Service (repeat session)

Thursday - 8/28/14

10:30-11:30am - STO1424 Massively Scaling VSAN Implementations: Another copresentation between Frans Van Rooyen from Adobe and myself on the early and rapidly evolving work being done to build and manage VSAN beyond single clusters and datacenters.

Thursday, June 26, 2014

#QRQ 3

My VM is locked up in VMware Fusion 6 and yet I can't restart the VM, because it's locked up:-)  How do I restart?

Hold down the (alt) option key when you click on "Virtual Machine" on the menu bar.  You should see the "Suspend" become "Force Suspend", "Shutdown" become "Force Shutdown", and so on.

Additional links:
http://kb.vmware.com/kb/1006215

Sunday, June 22, 2014

vCAC 6.x with Linux Catalog Items

Starting fresh to customize some Linux guest VMs with vCAC 6.0.1, I found that there was no simple authoritative source for doing so, at least according to several colleague queries and Googling so my apologies if I missed something obvious.  This also gives me my first chance to make a potentially [unsupported] recommendation.

I am working specifically with RHEL 6.4, CentOS 6.4, and Ubuntu 12.04 so of course using other Linux distros or versions, YMMV.  For each of those specific versions, I created a generic VM and mostly followed the defaults, installing the basic server packages and nothing else.  Feel free to customize your software and repos according to what you want in your baseline.  At the least, make sure you have Perl installed so that you can install VMware Tools and a "Core" or "Minimal" install does not even include Perl.  These were also created using static IPs.

After booting into the guest OS, install VMware Tools and I stick to the defaults here as well for each of the Linux distros and there shouldn't be much if any variance in the install options.  At this point, since we're prepping for vCAC, you most likely will benefit from including the gugent agent on your baseline.  The gugent allows callbacks to an agent service running on the deployed guest for additional configuration and customization.  However, this will vary depending on your Linux distro since some are supported, like CentOS, Redhat, and SuSE, but not Ubuntu.  Check
http://www.vmware.com/pdf/vcloud-automation-center-60-support-matrix.pdf, page 9, for details.

An excellent guide for installing the vCAC linux customization agent, and recommended blog to follow is here:
http://www.vmtocloud.com/how-to-create-a-vcac-6-linux-guest-agent-template/

If you are allowed to use DHCP, then that's pretty much it.  You can stop there and create individual blueprints in vCAC matching to each template.  However, for static IPs this is a little more unintuitive.  For Redhat and CentOS, you can follow these template prep guidelines:
https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.0/html/Evaluation_Guide/Evaluation_Guide-Create_RHEL_Template.html

Just leave out the first step:

# touch /.unconfigured

and follow steps 2-5 unless you want the user to have to work through network configuration dialogs every time a new VM is provisioned.  After deleting the HWADDR entry and if you have multiple eth# devices present, I have seen those sometimes getting reordered after reboot so be cautious if you are multihoming your linux VMs or have multiple private networks.  For step 4, I've been told you can also just delete the file:

rm /etc/udev/rules.d/70-persistent-net.rules

and this file will be automatically recreated after reboot anyway so haven't experienced any issues with that approach either.

Now right-click on the VM and, annoyingly enough, "Convert to Virtual Machine" isn't one of the obvious options.  You'll find it if you go to "All vCenter Actions"->"Convert to Virtual Machine".  Now I created a Linux customization spec with the following options:



Just use the virtual machine name and refer to your vCAC machine prefixes.  Incidentally, did you know that even though vCAC 6.x is stated to be multi-tenant, the prefixes are shared across tenants?
You're using UTC right?

Feel free to set to manual if you want, but the vCAC provisioning should override this to whatever static IP is available from your Network Profile.
Of course, make sure your DNS and Search Path reflect your environment.
Now when you create your blueprint, you can use this custom spec across your templates for CentOS, Redhat and Ubuntu right?




Redhat works, Ubuntu also works (though without a gugent for additional customization), but what about CentOS?  I would get an error deploying the CentOS template similar to the below, which was issued when just cloning direct from the vSphere Web Client:



 OK, so what may be unsupported is the workaround.  By setting the CentOS VM option to Redhat, this allows everything to proceed as normal and vCAC will deploy using the customization specification without griping.






Pretty straightforward once it's all in one place, hopefully:-)

Additional links:
http://www.vmware.com/pdf/vcloud-automation-center-60-support-matrix.pdf
https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.0/html/Evaluation_Guide/Evaluation_Guide-Create_RHEL_Template.html
 http://www.vmtocloud.com/how-to-create-a-vcac-6-linux-guest-agent-template/

Monday, June 16, 2014

VMware Office of the CTO, High Performance Computing

If you know me, then you know that I have a passion for HPC and virtualization and I enjoy a challenge.  I have been working with several customers behind the scenes on virtualizing HPC and developing this market for VMware and had presented at VMworld last year with UCSF on early work virtualizing their genome pipelines.  Now my latest challenge is to join full-time with VMware's Office of the CTO and I am humbled to be able to learn and contribute here.

I will be working with Josh Simons, leading HPC for VMware's OCTO and an HPC veteran and formerly with Sun Microsystems, towards advancement of how VMware approaches this market and the unique problems inherent in HPC as well as the problems now being shared by large "Web-scale" distributed systems.  It was at Hadoop Summit 2 weeks ago that I saw a lot of parallels resolving between HPC and the Hadoop ecosystem.  The common problems are classic computer science issues such as resource management and utilization as well as scheduling at different layers of the system.

And of course all of this is moving so fast, it's hard not to get distracted.  Which is why I appreciate being able to focus on this space because I will be able to leverage my background in compute, network, and storage as well as work on the bleeding edge on integrating and optimizing those with next-generation applications and frameworks.

Thursday, May 29, 2014

Quick Random Question #2

In a high latency, low bandwidth ROBO deployment scenario, how can a customer optimize their ESXi image for deployment so that the remote site will not suffer due to VIB deployment, function independently and allow for unreliable connectivity?

For starters:
1. On a vanilla ESX image, run "esxcli software vib list" to get the baseline of included vibs.
2.  On one of their prod ESX images (clustered, updated, drivers, everything to match what a prod host will look like at the remote site), run "esxcli software vib list" to get all the vibs they'll need.
3. Use ESXi image builder to create the software depot you need and they can export an iso to boostrap any remote hosts.






Additional links
Installing patches on an ESXi 5.x host from the command line
http://blogs.vmware.com/vsphere/2012/04/using-the-vsphere-esxi-image-builder-cli.html

Hadoop Summit and Hadoop as a Service

In the beginning of this year, I mentioned working on Big Data and virtualization and it has been a fruitful time.  Next week I will be co-presenting with Chris Mutchler from Adobe on "Hadoop-as-a-Service for Lifecycle Management Simplicity" at the Hadoop Summit conference in San Jose, CA.  Our session will be on Wednesday from 4:35pm-5:15pm.

I am humbled and excited to help present alongside other sessions from some of the most respected names in the industry from Yahoo!, Google, Cloudera, Hortonworks, MapR, Microsoft.  The growing depth, evolution, and community of the Big Data ecosystem is impressive, to say the least.  I hope to attend other Hadoop customer sessions as well as investigate what other large players are accomplishing from their respective stacks.  I see a lot of advanced sessions around new use-cases for Hadoop and research of adding additional layers and abstractions to Hadoop.  The Adobe session is focused on the usability of Hadoop from an IT operations perspective with a few key points to make:
  • Explain why virtualizing Hadoop is good from a business, techincal and operational perspective
  • Accommodate the evolution and diversity of Big Data solutions
  • Simplify the lifecycle deployment of these layers for engineering and operations.
  • Create Hadoop-as-a-Service with VMware vSphere, Big Data Extensions, and vCloud Automation Center
Hadoop is truly becoming a complicated stack at this point.  After I started actually getting hands on and working with customers on Hadoop-specific projects in 2011, I found that calling this new technology Hadoop seemed a bit disingenuous.  There was really MapReduce and HDFS, a compute layer and a storage layer.  Even though they were tightly coupled, that was enforced for very good and simple reasons.  Spending even more time on this has given more perspective on the different layers and their corresponding workloads.  Unless you're only running one type of job for your compute layer and sucking in data from a static set of sources for your data layer, then these workloads will vary as well as vary independently.  However, in a physical world, with both layers exactly coupled, how can they scale independently and flexibly?

Enter virtualization and everything I've been working on around virtualizing distributed systems, data analytics, Hadoop, and so forth.  Consider the layering of functionality for different distributions and look for the similarities.  If you take a look at Cloudera:


Or Hortonworks:
And Pivotal HD:

As a wise donkey once talked about in a movie when describing onions and cakes, they all have layers and so does any next-gen analytics platform.  Now we have our data layer, and then a scheduling layer, then on top of that we can look at batch jobs, SQL jobs, streaming, machine learning, etc.  Many moving parts and each one with probable workload variability per application, per customer.  What abstraction layer helps pool resources, dynamically move components for elastic scale-in and scale-out and allows for flexible deployment of these many moving parts?  Virtualization is a good answer, but also one of the first questions I get is "How's performance?"  Well, I have seen vSphere scale and perform next to baremetal.  Listed below is the link to the performance whitepaper detailing performance recommendations that have been tested on large clusters.

Speaking of all these layers, this leads to complexity very quickly so another angle specifically to the Adobe Hadoop Summit presentation is around hiding this complexity from the end-developers and making it easier and faster to develop, prototype, and release their analytics features into production.  Some sessions are exploring even deeper, more complex uses of Hadoop and I am eager to see their results, however, enabling this lifecycle management for ops is essential to adoption of the latest functionality of any vendors' Big Data stack.  VMware's Big Data Extensions, and in this case with vCloud Automation Center, allows for self-service consumption and easier experimentation.  There's a (disputed) quote that has been attributed to Einstein that states "Everything should be made as simple as possible, but not simpler."  There are a few vendors working on making Hadoop easier to consume and I would argue simplifying consumption of this technology is a worthwhile goal for the community.  Dare I say even Microsoft's vision of allowing Big Data analysis via Excel is actually very intriguing if they can make it that simple to consume.

Another common question I get is "Virtualization is fine for dev/test, but how is it for production?"  First, simplicity, elasticity, and flexibility are even more important to a production environment.  And maybe more importantly, let's not discount the importance of experimentation to any software development lifecycle.  As much as Hadoop enterprise vendors would like to make any Hadoop integration turnkey with any data source, any platform, any applications, I would argue we have a long way to go.  Any innovation depends on experimentation and the ability to test out new algorithms, replacing layers of the stack, evaluating and isolating different variables in this distributed system.

One more assumption that keeps coming up is the perception that 100% utilization on a Hadoop cluster equals a high degree of efficiency.  I am not a Java guru or an expert Hadoop programmer by any means, but if you think about it, it would be very easy for me to write something that drives a Yahoo! scale set of MapReduce nodes to 100% utilization but which really gives me no benefit whatsoever.  Now take that a step further as that job can have some benefit to the user, but still be very resource inefficient.  Quantifying that is worthy of more research but for now, optimizing the efficiency of any type of job or application specification will allow better business and operational intelligence to an organization and actually make their data lake (pond, ocean, deep murky loch?) worth the money.

Add to these business and operational justifications the added security posture:
http://virtual-hiking.blogspot.com/2014/04/new-roles-and-security-in-virtualized.html
and now you should have a much better idea of the solutions that forward-thinking customers are adopting to weaponize their in-house and myriad vendor analytics platforms.

Really exciting tech and hope to see you next week in San Jose!


Additional links
Hadoop Summit:
http://hadoopsummit.org/san-jose/schedule/
http://hadoopsummit.org/san-jose/speakers/#andrew-nelson
http://hadoopsummit.org/san-jose/speakers/#chris-mutchler
Hadoop performance case study and recommendations for vSphere:
http://blogs.vmware.com/vsphere/2013/05/proving-performance-hadoop-on-vsphere-a-bare-metal-comparison.html
http://www.vmware.com/files/pdf/techpaper/hadoop-vsphere51-32hosts.pdf
Open source Project Serengeti for Hadoop automated deployment on vSphere:
http://www.projectserengeti.org/
vSphere Big Data Extensions product page:
http://www.vmware.com/products/vsphere/features-big-data
How to set up Big Data Extensions workflows through vCloud Automation Center v6.0:
https://solutionexchange.vmware.com/store/products/hadoop-as-a-service-vmware-vcloud-automation-center-and-big-data-extension#.U4broC8Wetg
Big Data Extensions setup on vSphere:
https://www.youtube.com/watch?v=KMG1QlS6yag
HBASE cluster setup with Big Data Extensions:
https://www.youtube.com/watch?v=LwcM5GQSFVY
Big Data Extensions with Isilon:
https://www.youtube.com/watch?v=FL_PXZJZUYg
Elastic Hadoop on vSphere:
https://www.youtube.com/watch?v=dh0rvwXZmJ0