So I’m sure people are still wondering how far are we exactly?

The administrative system everything necessary to keep data and track of everything is in place.  The client area which shares he code base from some of the admin side is lacking the same thing the administrative system does not yet have.  This is the layer where communication to panels happen.  This means the abstract as well as the actual Openvz platform.  Besides that everything is pretty much in place though.  This is obviously a key to the system although I’d say having data do what it’s suppose to is a lot more important in the end than having the communication layer.  Communication layer can be changed/fixed but if data gets out of sync or is incorrect that’s a little more difficult to fix later on.

So to summarize everything been some delays but things are progressing just not at the rate I had hoped when we started.  It should start picking up as everything falls in place at Hawk Host to deal with the increasing number of new subscribers.

If you think you’re the kind of person we’re looking for then get in contact with me Tony by emailing tony with the at hawkhost.com added in.

Status Update

Not much to talk about today unfortunately still working on building the main part of the system which is managing the actual virtual machines.  Should have something complete on that front by the end of the week

• You have your virtualization platform itself for example OpenVZ, Xen ect.
• You have fields that are available to that platform so kmemsize or whatever.  Each field is dynamically generated from the database and has the capability to do validation, filtering and obviously provide a pretty name and description
• For the communication to the platform you have your driver which will handle all the communication to the machine and it’ll also extend the main platform which will handle all database backend

Here’s a bit of a more detailed explanation of the platform system:

Vezoz_VM_Platform

Every single platform will use this as it’s base to develop further.  It’ll handle all database portions and communications back to the panel.  You can in theory build any platform you want as long as it has the basics.  So the basics would be the facility to start, stop, reboot ect.  So with this you could develop something like say Vezoz_Vm_Platform_Openvz or Vezoz_Vm_Platform_Xen.  You could also take another approach and rather than using the platform itself you could do Vezoz_Vm_Platform_Hypervm if you wanted to use the HyperVM API and it provided everything necessary to you.  With a bit of work I’m sure you could also be crazy and do something like say Vezoz_Vm_Platform_Softlayer.  Softlayer has an API so you could potentially fool the system into thinking it’s dealing with a virtual machine when maybe it’s dealing with physical machines via the softlayer API.

The plan to start anyways is to just have an OpenVZ version available but we will expand it out for various platforms or even various ways to communicate (maybe two versions of OpenVZ communication).

Status Update

Right now we have started working on handling of actual virtual machines as we have every thing now in place for creation of them in the database end.  Hopefully before the end of the week we have a working system that will have all the necessary features for a first release.  Then the week after the plan will be to clean it up and have the Hawk Host user base using the first release.  Once that happens we’ll work on refining and adding features along with licensing capability for a release in the very near future.

The biggest worry among people is that the master has root access to all the slave servers in order to perform commands.  If it does not have access you then need a daemon of some kind to take commands and execute them.  With this method you then rely on the daemons being up to date (latest version) and having all the right commands available to be executed by the master.  Even with this method if a malicious user gains access to the master they can issue a  mass delete of all VM’s on the node which is the big scare everyone has.  This unfortunately is not possible to solve because if you give access to run vzctl commands you run this risk.  Everyone is suggesting using daemons even though it does not solve the fear anyways thus over complicating things.

The SSH protocol is a secure protocol and has capability of doing keys.  What many seem to not be aware of is the fact it also have the ability to restrict IP access on a key as well.  It even also has the ability to restrict commands but it is down to the parameter so that’s not an option.  The IP restriction though is useful because you can restrict the key to just the master’s IP.  So the only risk you now have is your master becoming compromised and a malicious user executing commands on the nodes.  Now doesn’t that sound familiar?  This is the exact issue a daemon service as that if the master is compromised the nodes are as well which makes sense.  The only difference is the fact with SSH you could wipe the entire system rather than just the virtual machines.  But at this point I’d say this is a small difference because if the virtual machines are gone the system is essentially worthless anyways.

Based on all of this I think we’re going to use restrictive SSH to accomplish everything.  There does not seem to be a necessity to have a daemon to handle the commands.  We can issue all the commands from the master we’ll just need to make sure with commands that take a significant amount of time to not execute them within anything web viewable.  So the best way to probably approach this is when a command like a OS rebuild is issued it will be executed by a cron within one minute.

Hopefully this provides some insight to what we’re doing and why.

$sql = “SELECT * FROM users”;

It will be something like this:

$select = $db->select()->from( ‘users’ );

The reason we do this is it provides the facility to mutate the queries very easily.  I’m building a query that has a lot of data and it’ll be displayed as a list paginated that means I need to calculate the total number of rows possible.  Most people would have to write two queries to handle this.  We do not as passing the select object into the pagination system it will drop unnecessary fields and replace it with a count.  This is great we’ve now only written one query to handle two situations and if it’s changed over time no worries it’ll be changed in the count portion as well.

So why does this matter?  Well I was building a query that in order to work properly needed to grab a set of rows and also join on another query thus creating a sub query.  As much as I tried I was unable to get Zend Framework’s Zend_Db_Select to work without error when a join is done on a query rather than a table.  A quick search I was unable to find a solution to this problem so as it stands it’s just a plain old sql query.  There isn’t too much of a negative to this it still passes the same sort of object back and if I can determine how to make it work later it can be switched thanks to it being part of a model.  So the code moves along for this portion not knowing better but it would be nice to eventually have it switched.

Status Update

Yesterday I talked about how the IP management system was nearing completion.  Well I am happy to say this has been completed and as far as having all the features necessary it is completed.  Usability will be improved but the goals right now are to make things work even if it’s a little more time consuming then what a final version will be.  We liked the HyperVM handling with ip pools so we found no reason to stray from the concept.  HyperVM however did not keep track of individual IP’s when they were not assigned to virtual machines while we do.  So here’s the steps on what goes on:

• You create an IP Pool and assign it a name along with network properties that apply to the entire pool (netmask and gateway which will be needed for Xen support)
• You add machines to the pool which will be able to use the IP’s.  The concept is machines on the same VLAN with IP’s routed to the VLAN should be able to use IP’s from the same pool
• Add IP’s to the pool individually or by range.  So you can just put 1.2.3.4 and add it or optionally do 1.2.3.6 and get 1.2.3.4, 1.2.3.5 and 1.2.3.6 all added to the pool.

After this the system keeps track of the IP’s which are available which are not as each IP is individual and can easily be linked back to a virtual machine.  Other concepts we’d like to add later on are the following:

• Ability to designate an IP to not have bandwidth tracking.  The idea behind this is our provider gives us NAS, iSCSI ect. which are through a private network
• Track bandwidth usage by individual IP rather than pooling it all together for the user.

Both of these depend on our way of tracking bandwidth usage but OpenVZ anyways we can do it.  With Xen not so sure as Hawk Host does not use Xen so our goal is OpenVZ first.  I am sure there are other idea’s floating around about what could be done but that’s it for now on the IP front.

The other thing being worked on right now is the operating system management portion.  We’re going with the concept of there being categories of operating systems as well as operating systems themselves.  The reason being for the user you could display them OS’s that are strictly that and you could also display them ones with control panels.  Of course the ability to make the OS not public to the user is possible as well as completely disabling an OS from being used again are possible.  This portion should be completed early on today then onto creating virtual machine plans and virtual machines themselves.  I will talk tomorrow about how we’re adding the concept of plans and how it varies from others.

Database Schema

The basics of the database have been designed to handle what we need from the start.  This includes but is not limited to machines, ip pools, ip addresses, virtual machines, users, user sessions.  This also of course includes relationship tables like how to relate an IP pool to a machine so that it can be used by that machine.

Framework

The beauty of this is with all our other projects already using the Vezoz name sake we have a lot of code developed for generic tasks.  We obviously use Zend Framework but that does not cover everything we have.  We a very complete and powerful model extension to handle the majority of data handling.  We have systems in place to quickly create listing pages that are very powerful (search, sort, pagination ect.).

Admin Area

Obviously the authentication system is in place for administrators to login.  It also now has the machine handling code in place as well as the majority of the IP address system.

Client Area

Nothing as of yet

Virtual Machine Handling

This being an obvious major piece of the software we’re currently determining the syntax and the commands necessary to replicate the old HyperVM system we used.  So basically all the vzctl commands as well as a few others used for monitoring.  We have determined how to handle traffic measuring and we will also be able to create IP’s that are not bandwidth tracked using our method.  This should be very useful for those who have private networks where they may wish to offer users NAS, iSCSI ect. but do not want that traffic to be counted towards the users bandwidth amounts.

Goals For Today

Today the goals are to continue to working on the administrative system.  The goal is by the end of the week at the latest have everything in place on the admin side that it will handle everything necessary on the database end to keep track of everything.  This could include it issueing commands that return as successful but really not doing anything (just calling the objects and methods that would do vzctl functions)

Why?

With LxLabs uncertainty due to the death of their founder as well as the current bugs in their HyperVM software we’re left with not many choices.  There are other pieces of software available now which are not near production ready in any form.  They are in most cases are just as uncertain as HyperVM with no actual developers.  There are others which are currently in development that we have no idea about their experience or how the software will turn out.  These new pieces of software are also months away from being available according to developers.  The other option is switching virtualization platforms which is also not an option

Why the Name?

Vezoz has always been our brand for anything development based.   Most of which never sees the light of day outside of Hawk Host but this will probably be the first to appear outside of our scope.  So with the trend of names going around we just added VM to the end to create the name of the software

Time Frame?

Our goal is to get it out at least in our use as fast as possible.  We’ll see how long that takes

Technologies Used?

While a lot of them choose to build daemons on the nodes themselves we see this as an unnecessary step which we believe leads to more problems then it solves.  We’ll see but our goal is to make that  not necessary and use a key system to access the actual machines as all commands issued for managing VM’s can be done without the necessity of a daemon or having the software installed on every machine.  We will be building thus upon PHP along with MySQL although other database engines should work due to the framework being used so PostgreSQL could be an alternative for the database side.

More

I have more to talk about but that’s the questions I wanted to get out of the way.  First of all we’re going to be doing rapid application development here rather than using a lot of planning.  The base of the system is very simple to design and we could have the frontend within a week and the management portion up within two weeks.  After which point the majority of the work will be building out additional features that are not necessary but will be nice.

With a lot of projects you wonder about the experience of the people behind it.  I do not claim to be some 20+ year expert but I do have experience in the real world and at one time my goal was to be a software engineer.  While still in school I did spend a total of a year over my time as a co-op working as a software developer gaining valuable experience.  I never completed this goal due to the time demands of Hawk Host Inc.  I’ve been doing PHP off and on since I was 16 (23 now) and have developed numerous applications during that time for various businesses.  PHP is not the only language though I’ve worked with C, C++, ASM, Java, C# (windows applications), C# (with asp.net) Perl and have various useless expertise in libraries with them.  For example I have experience using openGL mostly with games although I’ve used it for other things as well.

So lets see how things go over the next few weeks to a month and with almost daily posts I’m sure it’ll become exciting very fast.