The Imp must talk to its “Agent” in the Imp cloud before it can talk to anything else. There are many administrative reasons for this, but for the developer, the Agent is a very powerful programmable service. As an avid developer myself, I would encourage you to investigate it further before moving on. The agent can easily be programmed to talk to an outside server.
This forum is about the Electric Imp. All of us that use the Imp have signed-on to the concept of the “imp cloud”, agents, and hardware requirements.
But you should know, and already know, there are other devices available …
A more expense method, and a bit more programming needed, would be the FlyPort: http://www.openpicus.com/
The FlyPort is its own server. You actually create the webpages and hardware interface right on the device itself.
All of these WiFi control devices, and there are more coming out all the time, have their own specific “niche” in the control world. You have to decide based on features, ease of use, ease of upgrade/maintenance, and of course, cost.