[feature-request] VirtualBox Networking Model - new idea

[Technologov]

• Why is the TAP driver is "so needed"?

You say that Linux has something like this already, so you provide it only for Windows.

1. Actually, all Windows NT-based systems has something like this included.

The MS loopback can be added via Control Panel/Add Hardware/New Hardware/Manual/Network Adaptors/Microsoft. It is treated as just another NIC. Does your TAP driver provide you with features that you *absolutely* must-have? If not, then it's better to use the MS one, because it's signed. (and gives no stupid driver signature warnings during adding of an extra NIC)

---

Actually, I think we can get much more far *without* any such TAP drivers at all.

---

2. I think that the current networking model needs new revision: (if I

understood your networking model correctly) : it can be much more scalable, if it will be done the "Microsoft Virtual PC 2004"-way; (or if you hate MS way, then look at the Open-Source emulator "Dynagen"-way)

that is: I think that we don't have to use any such TAP driver at

all. We can use the host network interface directly - of course we need some way to enumerate them - two ways exists: Windows Registry or WinPCap. (BTW: Open-Source Wireshark, while not an emulator, also uses this technique)

With this approach hubs will be created automatically, without any need to define bridges manually. Windows 2000 host will be supported much better this way (because AFAIK it doesn't have bridging) and Windows XP host won't be limited to one bridge as it is now.

As a bonus, the Linux host networking will become piece-a-cake - instead of manually creating TAP interfaces and then bridging them to real host interfaces (manually again, all via command-line), this new approach will enable to integrate it all nicely within a GUI so click-click-click will solve it all. Basically a Linux user, working with a GUI-only, will be able to achieve host-networking effect under Linux very easily !

The possible downside costs is an extra dependency: libpcap/WinPCap (to make things cross-platform).

Please be patient, because I might be incorrect in some cases; after-all I looked at your software for only one day. Please tell me where I'm right and where I'm not.

-Alexey Eremenko (CCNA, Cisco Certified Network Associate)

(originally sent :vbox-users@…

date Feb 7, 2007 10:49 PM)

[Klaus Espenlaub]

The loopback adapter is no suitable replacement for the TAP driver. Loopback+sniffer (pcap) is inferior to a bridge (which delivers traffic only to the NIC with the matching MAC address). So what you propose is actually more complicated, less versatile and less scalable. What's also ugly about the pcap approach is the fact that some antivirus/antispyware complain about sniffers being active.

All this trouble only to solve a non-existing problem (on Windows 2000 host one can still use routing instead of bridging, it's just not described in the manual). The way we do it might not the way everyone else does it, but it looks quite logical to us. In the real world you also don't have Y cables which connect to multiple NICs, you have a switch in between (and "bridge" is just the outfashioned network terminology for "switch").

And by the way, Windows XP is not limited to one bridge. One create as many bridges as one wants. However as in real networking, a particular NIC can only be attached to one bridge at a time.

[Technologov]

OK, feature-wise, what you need from the TAP driver that isn't included with MS Loopback?

[Klaus Espenlaub]

The most important feature is the ability to send/receive ethernet packets as directly as possible from a normal user-mode application. Ideally without requiring admin rights. The MS loopback adapter is just a special network card with no physical network, and as a result we still need winpcap to send/receive packets (because one needs a special NDIS driver for that, which WinPCAP provides). So it seems to me that the only user-visible effect is replacing WinTAP with WinPCAP (the remaining differences are big documentation changes). Correct me please if I'm wrong.

[Technologov]

You're solution is bad.

It is both complex, poorly portable and not User-Friendly.

Why is the routing on Windows 2000 hosts isn't included in the manual ?

Let me answer for you: because this trick is more complex than simple bridging and doesn't provide for simple and transparent, GUI based solution.

I know plenty of tricks that can work with simple NAT, that will allow me to access a firewalled computer as a server that are based on tunneling approach (OpenVPN, Hamachi VPN, L2TPv3, Tsocks, ...), but all those are just that – dirty tricks.

You told me than Windows XP supports more than one bridge ?

Wrong again ! Check your facts. Let me help you find the docs on your XP system: Start->Help and Support->search for “bridge”->Network Bridge Overview.

There you will find a string near bottom of document: “Only one bridge can exist on a computer running Windows XP”

And if you don't believe MS docs, you can check this: just create 4 loopback devices, accroding to my method, and try to add them to 2 virtual bridges, 2 NICs (loopback devices) in each virtual bridge.

Let me save your time: Impossible. You either get all 4 in one bridge, or 2 in bridge, and 2 not bridged.

Basically, this means that I have no simple/good way for host networking on Windows 2000, and a very limited way to do this on XP. (just 1 NIC can be connected securely to the remote world).

(That's not suitable to test enterprise Firewalls for example, that must connect to many physical ports at once). I do complex network simulations, and simulating dozens of machines and hundreds of NICs same time, but a simple way must exist.

As for performance:

On real hardware, bridging or switching might perform better than hubs due to division of collision domains, but in the virtual space it is quite the opposite: virtual hubs often outperform virtual bridges or switches due to lower CPU overhead (and have better stability due to simpler architecture).

I have no numbers, but building a test case for libpcap vs. Win XP bridging might be interesting.

Ease-of-Use:

The current Host Networking cannot be done via VirtualBox GUI, but rather depends on the platform to do that, so read on.

Portability: (Cross-Platform aspect)

The current solution is highly dependent on the platform to do bridging. If the platform has no such feature (Windows 2000), then it doesn't work.

If the platform has no GUI for bridging (Linux) then I will have to use command-line for that. Basically, the current model forces me to use command-line for Host-Networking on Linux.

Worse yet: if some Linux distro doesn't provide bridging-tools ? then it will be like Win2000, unable to do Host bridging at all.

The new solution must depend on libpcap to do the job, and our GUI to connect to it. It is the portable way of doing things. The new model will enable GUI-based Host-Networking independedly of the host's willing to support bridging.

Addition scalability problem:

creating a virtual NIC on host system for every host real NIC for every connection I want to make through Host.

That is, if I have a 4-port firewall (4 NICs) on guest, and 8 NICs on real host, after connecting to host I will have 12 NICs on Host. (8 real NICs and 4 TAP interfaces)

Since I cannot create 4 bridges (one allowed under Win XP, and zero under Win 2000), we get back to the portability problem.

I have not recommended using MS Loopback with WinPCap, but rather a possibility to replace current TAP driver with MS Loopback (as it can simulate ethernet NIC or hub and even has ability to change MAC addresses – a needed feature for VirtualBox).

The new solution scales to dozens of NICs easily (tested) and perhaps even hundreds. (will-be-tested) without adding any extra unneeded NICs (TAP interfaces) to the host.

Summary - The current solution is bad, because hosts either do not support bridging (Win2000), or support only 1 bridge (WinXP), or require command-line (Linux).

The recommended way:

We need a good host-networking solution, with the following features:

-Easy to understand

-Cross Platform

-Easy to Use (with a GUI)

-Scalable

-Stable

I see only libpcap/WinPCap as a good contender.

Our Qt3 GUI will need just small changes, so it will be able to enumerate the host interfaces correctly and attach to them directly. (without the need to create MS Loopback, TAP interfaces, doing bridging by using Host tools, and other stuff like that).

This solution will effectively convert any Host interface (real or loopback) into a virtual hub. So the stability, scalability, performance, easiness of understanding, portability and User-Friendliness will all improve. This works like a hub. libpcap allow to connect many guest systems (even from different emulators) to one single NIC, and all can send and receive data. Additionally, an external device (real remote computer) and our host computer can also send data to such a hub, making this ideal connectivity solution.

And last but not least: If you really want the current networking model to stay, OK, so be it, leave the current and add a new mode, but we *must* implement an alternative networking model based on those well-tested concepts, that I just described.

-Alexey Eremenko (CCNA, Cisco Certified Network Associate)

[Achim Hasenmueller]

I agree with most of what you say. However, we believe TAP is the best solution on Linux because it is always preferable to use a standard kernel feature rather than a proprietary module or interface. TAP is the most flexible method on Linux. The fact that it's rather complex to setup is more a Linux problem than VBox problem.

On Windows, we have always planned to look for a replacement using something like an NDIS wedge / filter driver.

[Klaus Espenlaub]

I hope you accept my apologies. Didn't mean to brush you down with incorrect facts. Verified myself that you can only have one bridge with WinXP. My memory has deceived me apparently (I thought I saw a WinXP system with two bridges configured). We're looking into how we can implement this. However it'll take some time/effort to implement, because we'll need some solution which works for Vista, too. We have to move away from requiring Admin rights anyway, so the sniffing needs to go into some service which has the necessary privileges. We don't want to open big security holes. The user interface won't need to change much, that's for sure.

However I don't agree that routing is a "trick". The reason why it is not in the manual is that it's impossible to explain how to set up routing generically in a reasonably short way. It depends too much on what the user wants to achieve, what network topology he already has and so on. But for you as a CCNA this is probably nothing new. Keep in mind, VirtualBox is rather new to the end-user world. So far we only dealt with relatively big customers, and that's why we chose the "contact us" way in the manual. Not to annoy normal users.

[Michael Thayer]

Alexey, please correct me if I am wrong here. It seems to me that on Linux, libpcap sends to and receives from the network interfaces at the package level, which requires administrator privileges. If so, it means that either the main VirtualBox process must be running as root, not something which we really want, or we need a separate daemon to handle the network part, running as root and communicating with VirtualBox processes through some form of IPC, which might adversely affect network speed.

[Technologov]

to michael and all: You are correct, libpcap requires root because you're accessing host-hardware directly, so potentially this is a security hole. (if it would not require root, everyone could sniff host-interfaces...)

Unfortunately, I'm non-programmer and not a security expert, but still will try to explain a possible workaround:

maybe setuid/setgid for libpcap access is the best way(?), so only "vboxusers" could access it. I mean, that only users who are part of "vboxusers" group must have access to host-networking. All non-privileged users won't have this privilege.

-How is this idea ? Any better ideas or workarounds? Will this create a major security hole, or an acceptable risk ?

We should find this out from people who understand libpcap better than me. Perhaps, implementation details could be asked directly from mainstream, from libpcap developer's team.

-Alexey Eremenko

[Michael Thayer]

Unfortunately, there is no way to give a library different privileges to the application using it. One workaround to this is what I described above - split off the part of the programme which uses the library into a separate process (or better, a separate programme), give that process the required privileges and have some way for the two to communicate. However, in practice that would mean every network package in the VM going through VirtualBox, our privileged networking daemon and through the host kernel (which has to pass the package between the two using IPC). For me at least, it is hard to judge how much additional overhead this would imply.

[alvarezp]

*PLEASE*, I think TUN/TAP it's ok. Changing it to the way it's proposed, you wouldn't be able to do any routing/briding tests flexibly. It's also impossible to make a hub act like a switch, but it *is* possible to make a switch work like a hub.

Now, I *do* think than although it's fine the way it is now, it sure needs a better GUI for TAPping. Maybe provide some sample scripts and fine-tune the docs.

Just an opinion.

[rafelbev]

I too believe that the current modus operandi is a good flexible solution. I place a vote too for keeping the current networking model, but improve drastically the GUI to make things just work.

I manage to do everything via command line... however I do understand the people who find such a task too daunting.

[JW]

Ok, am I seeing this correct? You can go directly to the internet, but when you try to setup the TAP to be able to access folders of other computers on the local area network it just wont do it?

I'm having that same problem. Has anyone found an EASY way to do this yet? The reason I wanted VB is to be able to run XP in it and transfer files to the other computers on my network. And in this way, I can test files for spyware etc. If something is worthy of keeping then I can transfer it to the other computers via file sharing on my personal network.

If I missed something can you please update me?

Thanks for this great product and your time. JW

[Michael Thayer]

Besides shared folders, you can easily copy files between guest and host using openssh. This can be done using NAT networking in the VM. Simply set up an openssh server on the host (e.g. Debian or Ubuntu host: apt-get install openssh-server). This should not be a security problem as long as the host is behind a firewall or a NAT router (most home routers are NAT) which does not forward those ports. Then install WinSCP (Windows) or scp (Linux, part of the openssh client suite) on the guest. winscp or scp can then copy files to and from the host.

[JW]

ok, here is a Q for ya.. I downloaded Putty 0.95. I guess i'm supposed to install it to the HOST machine. However I can't figure it out. (Please forgive me, I have no clue as to how to work with networks other then the SIMPLE WinXP folder sharing on 3 of my computers.) I want to share between all three computers and still be able to get to the internet with all three computers. I have all of them hooked up to a router. AND I can get the Virtual Box hooked up the the internet directly. But what I don't know is what to do get all computers to be able to talk directly to the VB. The program is the best one I have seen. (LOTs BETTER THEN MS version!!) And please do not get me wrong, i'm not complaining. For free, I have no room to complain. But can someone write out a step by step explanation as to how to get Putty (or any other Win XP software) installed to accomplish this task? I really NEED to be able to move files from the VB to the other computers on my network (As well as the Internet.).

By the way, this is Soooo cool... I never really messed with Virtual Software before. And the idea of my making my music on the VB and moving them to another computer as a backup really appeals to me. (Same thing with my 3D art that I love to create for myself.)

Thank you for your time folks.

JW

P.S. If Putty is not what I need, can someone tell me what I do need? (Step by step installation please.) Thanks!!!

[Michael Thayer]

I take it that you are running a Windows host, since you are looking at installing Putty on the host. I have never tried installing an ssh server on a Windows machine, but looking at the openssh homepage, cygwin + portable openssh looks like the way to go. I presume that it is not difficult (it is simple on a Linux host), but can't tell you offhand how to do it.

I think though that this is getting a bit off-topic for the feature request - you might want to take your questions to the mailing list (or for setting up an openssh server, possibly to the openssh mailing list or forums). Alternatively, you could look at VirtualBox shared folders, which do however still have a few glitches on Windows guests.

[JW]

I'll look into it. Thank U for your kind response.

JW

[Technologov]

Now, please cleanup all the bull from this bug. plz delete last few posts - starting by 28.02. by JW, including this one.

-Alexey

[Technologov]

Hello !?!

Is this bug moving anywhere?

There are many people complaining for poor networking support.

Tip: Dynagen/Dynamips has switching functions emulation on both Linux and Windows. You can check it out. (tips goes both to advanced users and to developers)

-Alexey "Technologov"

[Achim Hasenmueller]

Uvazhaemyj Alexey Technologov,

we as InnoTek are not going to change our networking model on Linux. Maybe make a few things a bit easier. We will eventually do a type of protocol driver on Windows to make it easier to configure and eliminate the one bridge limit of XP/Vista.

In general, our agenda is pretty full. We are more looking for sensible code contributions than repeated requests for things we have neither resources no motivation to do.

[Michael Thayer]

Alexey,

We are not going to be adding this functionality ourselves, since no paying customer is requesting it, and it is too large to add without that. However, since VirtualBox is open source, others would also be able to do that for you. If the feature is of sufficient interest to you, you could look for someone external to InnoTek willing to do it either for free or for a fee. You might also want to look for other people interested in the feature and willing to either help pay for it or contribute part of the code. If it integrated well enough with our existing code base, we would probably be able to integrate it into the released version of VirtualBox.

For the reasons I highlighted above, it would probably be best to write most of it as a separate privileged daemon process which would access the network devices directly and exchange network data with VirtualBox. This model would also be easier for someone external to write, as it would require less familiarity with the VirtualBox sources.

[Technologov]

Some link on how Qemu does this: ​http://qemu-forum.ipi.fi/viewtopic.php?p=5646&sid=494a2df12b48a1e8fdca8ae0e7bfb752

In short, using pcap is possible alternative to TAP and bridging.

-Technologov, 27.7.2008.

[Technologov]

At last ! This functionality was added with new "vboxflt" driver in VBox 2.1.0.

Please CLOSE this bug as FIXED.

-Technologov, 17.5.2009.