×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Have Sockets Run Their Course?

kdawson posted more than 4 years ago | from the as-it-was-in-the-beginning dept.

Programming 230

ChelleChelle writes "This article examines the limitations of the sockets API. The Internet and the networking world in general have changed in very significant ways since the sockets API was first developed in 1982, but the API has had the effect of narrowing the ways in which developers think about and write networked applications. This article discusses the history as well as the future of the sockets API, focusing on how 'high bandwidth, low latency, and multihoming are driving the development of new alternatives.'"

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

230 comments

haha (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#27933851)

tiny penis

Really... (3, Funny)

Wingman 5 (551897) | more than 4 years ago | (#27933853)

I think sockets work fi.... *connection lost, host not routable*

Re:Really... (4, Funny)

rolfwind (528248) | more than 4 years ago | (#27934133)

I think sockets work fi.... *connection lost, host not routable*

Really, for networking, all they need to do is ask slashdot's elite technical team. Years before Gmail automatically saved my drafts, /. consistently preempted everone with the above example (or Homeland_Security/FBI/Police knocking on the door, or person getting a hard attack) and snatches the post from the jaws of defeat when the user wouldn't otherwise be able to hit submit. Moreover, unlike anyone else to this day, even gmail, there is also a nice little hint as to the cause of the interruption.

Re:Really... (5, Funny)

knutkracker (1089397) | more than 4 years ago | (#27934371)

or person getting a hard attack

Viagra overdose?

Re:Really... (5, Funny)

lskovlund (469142) | more than 4 years ago | (#27934625)

This is not funny. It's called priapism [wikipedia.org] and can result in impotence or worse.

Re:Really... (0)

Anonymous Coward | more than 4 years ago | (#27934937)

LOOOOL

I thought you were serious until I read "or worse".

Re:Really... (1, Informative)

Anonymous Coward | more than 4 years ago | (#27935619)

You think the "or worse" is a joke? Apparently you didn't read the link:

Potential complications include ischemia...the ischemia may result in gangrene, which could necessitate penis removal.

Re:Really... (0)

Anonymous Coward | more than 4 years ago | (#27934361)

he thnk

whats really needed... (2, Insightful)

Anonymous Coward | more than 4 years ago | (#27933865)

is no sockets. some way to seamlessly connect LOCAL processes to each other without socket overhead by using the familiar socket interface. something simpler than shared memory.
and a better protocol method of opening sockets with the hard stuff taken care of by the OS. and with transparent buffer protection etc.

Re:whats really needed... (5, Informative)

Anonymous Coward | more than 4 years ago | (#27933887)

You mean like this? http://en.wikipedia.org/wiki/Unix_domain_sockets

Re:whats really needed... (1)

i.of.the.storm (907783) | more than 4 years ago | (#27933901)

I'm seriously confused, was the OP AC aiming for someone to post that or were they just ignorant? Parent is right either way.

Unix always had it (4, Informative)

mangu (126918) | more than 4 years ago | (#27935255)

some way to seamlessly connect LOCAL processes to each other

You mean, like pipes [wikipedia.org]?

Open Transport, Part II (5, Informative)

Etcetera (14711) | more than 4 years ago | (#27933895)

Been there, done that. Apple (once again) had a great implementation of an alternative technology, that it finally abandoned when it didn't feel like fighting any more.

Open Transport [wikipedia.org] (the PPC stack used in the Classic Mac OS) was fast, efficient, and cool. And based on the STREAMS [wikipedia.org] methodology, the only real competition to Berkeley Sockets.

Choice is good, mmmkay?

Re:Open Transport, Part II (2, Funny)

Anonymous Coward | more than 4 years ago | (#27933975)

As next you will probably claim Apple has invented MAC addresses too....

Re:Open Transport, Part II (5, Funny)

Anonymous Coward | more than 4 years ago | (#27933997)

Well, I did hear it was a Xerox standard so it must have been copied from someone. I guess it could have been Apple.

Re:Open Transport, Part II (4, Funny)

Concerned Onlooker (473481) | more than 4 years ago | (#27934017)

"Well, I did hear it was a Xerox standard so it must have been copied from someone."

I hope you meant to make that joke.

Re:Open Transport, Part II (-1, Redundant)

Anonymous Coward | more than 4 years ago | (#27934041)

Well, duh.

Re:Open Transport, Part II (1, Insightful)

Anonymous Coward | more than 4 years ago | (#27934023)

Sorry to burst your bubble, but I remember STREAMS and they sucked. I was also one of the earliest Mac programmers, and frankly, there was little to like in Classic Mac OS either, other than a lot of fiddly and clever hacks to make it fit into 128k.

Yeah, Apple abandoned those technologies because they were tired of fighting alright... fighting their limitations and bugs.

Re:Open Transport, Part II (3, Insightful)

Anonymous Coward | more than 4 years ago | (#27934225)

Open Transport didn't come about until the mid 1990's.

So, if you were programming for the Classic Mac OS in the 128K days, still doing that 10 years later and hating it *that* much, you probably feel like you've wasted half your life.

Yes, you could have moved on to other, newer, more advanced operating systems, but you *chose* to stick with it. One really has to respect that I suppose.

Shows your more masochistic side.

Re:Open Transport, Part II (1, Insightful)

Anonymous Coward | more than 4 years ago | (#27935121)

What didn't you like about STREAMS? Was it the nice architectural way you could layer processes or the efficient way you could avoid data copies?

Re:Open Transport, Part II (4, Informative)

Shin-LaC (1333529) | more than 4 years ago | (#27935339)

I wrote networking code using Open Transport before I ever touched sockets, so I think I have a view as unbiased as you can get (or perhaps biased in favor of OT). I didn't mind OT, but I when I moved to sockets I was impressed with how easy and comfortable it is to work with them. The limitations of the classic Mac OS architecture probably made writing Open Transport code thornier than it would have been on a moden system, so I won't discount TLI or STREAMS in general, but I have to say that the particular implementation that was Open Transport on Mac OS is inferior to sockets on a UNIX system.

RFC 1925 (4, Insightful)

Endymion (12816) | more than 4 years ago | (#27933905)

This seems to dance a bit too close to Networking Truths [faqs.org] 6a, 11, and possibly 12. I will reserve judgment until I see solid real-world evidence.

Re:RFC 1925 (3, Interesting)

dbIII (701233) | more than 4 years ago | (#27934297)

There are some sitautions where it isn't the best choice. In very simple clustering they just may not be enough sockets. For instance one package uses "rsh" up to around 512 hosts beyond which it doesn't work reliably unless you use "ssh" and a single socket. Of course "rsh" access scares people for plenty of other good reasons but that's a point best discussed elsewhere.

Re:RFC 1925 (4, Insightful)

Endymion (12816) | more than 4 years ago | (#27934585)

Yes, there are always pathological cases that demonstrate the weaknesses of any technique. A big point I take away from RFC1925 (and personal experience), is that you have to A) recognize that trade-offs are always going to be made, and B) adapt your implementation to fit the laws of physics, instead of trying to bend the network to fit what you think an implementation should be.

The simple fact is that Sockets have worked very well for a long time. Yes, this sometimes means you have to shape your design and implementations to fit the "socket style", and history has shown that it is not only possible, but practical. Changing to a new design will not remove the fact that if you design your protocol/app badly, or are inherently in a pathological use-case, then your network performance will suffer.

For some problems, the ssh idea of multiplexing a single socket works well. For others, multiple rsh (*1) style work better. To say that Sockets need to be replaced because you chose to use rsh for your transport is an amazingly arrogant (*2) position. And yes, some of this is "tradition" and inertia, but designing a whole new library should be for significant real-world benefit, and not for corner-cases or maginal 1% gains.

Of course, if someone can actually produce some real-world benchmarks that validate the "let's ditch Sockets" claim...

[*1] As with you, this is totally ignoring the security implications, etc.
[*2] In no way is this a personal attack at you; I mean it in a purely academic sense. It's a very tall claim to say that decades of networking history, and thousands of talented engineers were wrong.

Re:RFC 1925 (2, Interesting)

dbIII (701233) | more than 4 years ago | (#27935211)

Yes, that's why I said "some". Just like the guys that wrote clustering software that is really just "rsh" and couldn't imagine anyone running it on a couple of thousand nodes it looks like the author hit a case where it really should have been done another way. Good answer above, however what I really was doing was trying to show a way that sockets can be used badly or used well.

Re:RFC 1925 (2, Interesting)

ThePhilips (752041) | more than 4 years ago | (#27935525)

Of course, if someone can actually produce some real-world benchmarks that validate the "let's ditch Sockets" claim...

There are really few real world example where you can do something better than sockets.

BSD sockets are quite versatile API. I have programmed them on both side - implementing my own protocol/address family and actually using them in program - and hardly see how one can do it better, maintaining level of guarantees provided by the API. And the level of guarantees what makes it possible to develop applications behaving reliably/predictably under ever varying conditions - and not loose your sanity in the process.

Also what many novice forget that sockets support a number of assertions application can make on sync/async error handling. IOW, one can easily improve performance of BSD socket by simply removing error handling. But something tells me that no-one's gonna do it.

Hilarious (5, Insightful)

karmaflux (148909) | more than 4 years ago | (#27933941)

This guy's worried about "narrowing the ways in which developers think about and write networked applications" in a world where people are reinventing wall(1) [ed.ac.uk] as twitter, IRC as friendfeed, and other web 2.0 'innovations.' You want to widen developers' thinking about networking? Leave sockets alone and close off port 80.

Re:Hilarious (1)

noidentity (188756) | more than 4 years ago | (#27934019)

Anyway, isn't "narrowing the ways in which developers think about and write [type of] applications" another way of saying it abstracted [wikipedia.org] things?

Re:Hilarious (1, Insightful)

Anonymous Coward | more than 4 years ago | (#27934449)

Twitter seems more .plan than wall... but I agree ;-)

Which sockets API? (4, Informative)

PhrostyMcByte (589271) | more than 4 years ago | (#27933943)

There are Berkeley sockets which are relatively portable, and then there are extremely platform-specific APIs for high performance and scalability. The old API might have run it's course, but most of the new ones are still relevant. Things like asio [sf.net] are helping to merge all the differences into one nice API.

Re:Which sockets API? (5, Interesting)

Anonymous Coward | more than 4 years ago | (#27935513)

The Berkeley socket API has stood up very well against the tests of time, and it is fairly lean and quite versatile, but yeah, there's definitely room for newcomers.

For example, when it comes to high packet rates - say, thousands of VoIP RTP streams - the length of the typical path a packet takes through the kernel layers becomes quite prohibitive.

I've been trying to reach gigabit ethernet saturation with G711 VoIP RTP streams (that is, 172-byte UDP packets @ 50Hz per stream), which works out to a theoretical maximum of 10500 streams - 525000 packets/second. My initial speed tests, with minor tweaking, got me around 1/10th of that, thanks to all the kernel overhead, and the lack of control over how and when packets will be sent.

So I wrote my own socket-> UDP-> IP-> ARP-> Ethernet abstraction which hooks directly into the PACKET_MMAP API (as used by libpcap), with the TX Ring [ipxwarzone.com] patch, and with all the corner-cutting I managed to achieve 10000 streams (500k packets/sec) which equates to about 95% of the theoretical peak.

In short, we probably need more widespread support for different network programming APIs which address more specific needs - BSD sockets are too generalised sometimes.

Re:Which sockets API? (1)

ThePhilips (752041) | more than 4 years ago | (#27935551)

Ignore the RTFA. Quote:

... the calling program must repeatedly ask for data to be delivered.

I presume that the date on the article is off by 10 years or something. I make the judgment based on the facts that the author calls SCTP "recently developed" and apparently never heard of /dev/epoll or kqueues (or e.g. libevent allowing to use them in portable manner).

wrong (4, Interesting)

jipn4 (1367823) | more than 4 years ago | (#27933981)

Although the addition of a single system call to a loop would not seem to add much of a burden, this is not the case

Really? For a lot of networking code that's in use these days, I don't see that the system call overhead is the bottleneck. On clients you usually have network bandwidth as the limiting step (rather than system calls). On servers, it usually seems to be disk access or HLL interpreters.

Each system call requires arguments to be marshaled and copied into the kernel, as well as causing the system to block the calling process and schedule another.

That's easy to fix without changing the socket API: just add a system call that can return multiple packets from multiple streams simultaneously, a cross between select and readv. If there's a lot of data buffered in the kernel, it can then return that with a single system call.

Solving this problem requires inverting the communication model between an application and the operating system.

Not only does it not require that, inversion of control doesn't even solve it, since you still have the context switches.

Re:wrong (2, Interesting)

jipn4 (1367823) | more than 4 years ago | (#27934055)

Oops... left out half of it...

That's easy to fix without changing the socket API: just add a system call that can return multiple packets from multiple streams simultaneously, a cross between select and readv. If there's a lot of data buffered in the kernel, it can then return that with a single system call. The user mode socket library can use that system call internally and still present every caller with the regular select/poll/socket abstraction; when callers request data, it first returns data that's already buffered in the process without another system call, and when it runs out of that, then it calls back into the kernel.

Re:wrong (5, Informative)

TheThiefMaster (992038) | more than 4 years ago | (#27934765)

Windows' solution is pretty nice. You can pass a pre-created socket handle to accept_ex, which automatically accepts an incoming connection using that socket handle, so that you don't have to use two system calls (select and accept). You can also pre-accept multiple sockets, instead of having to make the system calls under load.
Sockets can also be closed with a "re-use" flag, which leaves the handle valid and saves making a system call to create another.

You then associate the sockets with an "IO completion port", which as best as I can tell is a multithreaded-safe linked list for really fast kernel to user program communication. To receive from the socket you make an async receive call, giving a pointer to a buffer to receive into.
Whenever data is received on those sockets (and has had a corresponding async request made for it already) the kernel automatically queues the socket handle to that linked list. If you associate a socket with the completion port before you accept a connection with it (i.e. you're using acceptex) it also triggers when the socket accepts a connection.
In the user code, you run multiple threads listening on the completion port (you can also use the completion port in the thread pooling API, which runs two threads to each cpu core by default). When a message arrives from the kernel, the most recently finished thread wakes and processes the received data, which will already be in the user-space buffer you provided in the original receive call.

If all threads are busy and there are messages in the completion port they will bounce right off of the completion port, picking up the next bit of completed IO they need to process without making a system call.

Re:wrong (4, Interesting)

RAMMS+EIN (578166) | more than 4 years ago | (#27935205)

``Windows' solution is pretty nice. You can pass a pre-created socket handle to accept_ex, which automatically accepts an incoming connection using that socket handle, so that you don't have to use two system calls (select and accept). You can also pre-accept multiple sockets, instead of having to make the system calls under load.
Sockets can also be closed with a "re-use" flag, which leaves the handle valid and saves making a system call to create another.

You then associate the sockets with an "IO completion port", which as best as I can tell is a multithreaded-safe linked list for really fast kernel to user program communication.''

I don't know. To me, it all just sounds like kludges to work around the facts that system calls are slow and that the implementation of the Berkeley API causes many system calls. You are adapting the structure of your program to code around the problems, instead of fixing the problems that cause the natural style of your program to lead to slowness.

There is nothing in the Berkeley socket API that mandates system calls or context switches. At worst, some copying is necessary (because the API lets the caller specify where data are to be stored, instead of letting the callee return a pointer to where data are actually stored).

The reason we have system calls and context switches, I claim, is that we are using unsafe languages. Because of this, applications could contain code that overwrites other programs' memory. We don't want that, and we have taken to separate address spaces to avoid it. The separate address spaces are enforced by the hardware, but this has a price, especially on x86. Perhaps it is time to rethink the whole "C is fast" credo. As the number of work instructions that can be executed in the time it takes to do a context switch increases, so does the relative performance of systems that do not need context switches, but of course we can only do away with context switches if we can provide safety guarantees in another way. One way would be to have the compiler enforce them. But that is outside the scope of Berkeley sockets, of course.

Re:wrong (0)

Anonymous Coward | more than 4 years ago | (#27935587)

This isn't remotely feasible, and suggests that you don't understand Unix I/O very well (or at all).

For a start, a socket can be shared by more than one process, and you don't know which process should receive the data until the process actually read()s it.

Re:wrong (3, Insightful)

convolvatron (176505) | more than 4 years ago | (#27934083)

if this were all in one domain, the most flexible and efficient thing would be to have memory for receive frames allocated at the bottom of the stack, and use callbacks all the way up.

because of the user kernel boundary we have a copy which is difficult to get around (put the next 1k bytes exactly here, although i really dont care), and some unfriendly and inefficient hacks to weasel around the 'natural' blocking semantics.

even if its completely academic, i think its interesting to look at the user kernel boundary and try to refactor things which have negative structural impacts.
 

Re:wrong (4, Interesting)

jipn4 (1367823) | more than 4 years ago | (#27934179)

even if its completely academic, i think its interesting to look at the user kernel boundary and try to refactor things which have negative structural impacts.

And you think that 2009 is the first time people think about this? System call overhead used to be a much bigger issue. UNIX and Linux has the current set of interfaces because they are a good compromise between simplicity and efficiency.

And these issues are constantly being evaluated implicitly: people who write network servers benchmark their code and find the bottlenecks. If the bottleneck is some system call, they complain to the kernel mailing list and maybe roll up their sleeves and come up with something new. If that turns out to be useful, more and more people ask for it to be put into the kernel, and eventually it becomes standard.

What motivates kernel developers is real benchmarks and the needs of important, real-world applications, not fluff pieces that express generic displeasure with the way things are done.

Re:wrong (4, Interesting)

convolvatron (176505) | more than 4 years ago | (#27934223)

no. in fact i can remember having discussions myself about this more than 20 years ago, and those were hardly the first.

unix has these interfaces as a matter of historical accident, what was an excellent design at the time. its hardly the only good point in the space.

you might find that it helps to think about these thing..even when developing important, real-world applications. why shouldn't the kernel be able to call into userspace safely and transfer ownership of a buffer? is that really so terrible to consider?

Re:wrong (4, Insightful)

jipn4 (1367823) | more than 4 years ago | (#27934475)

unix has these interfaces as a matter of historical accident, what was an excellent design at the time.

No, UNIX has these interfaces because they get the job done. People tried all sorts of other interfaces and none of them caught on.

you might find that it helps to think about these thing..even when developing important, real-world applications.

How does it "help" me to think about solutions to problems I'm not having? I've never seen the socket interface to be rate limiting in anything I care about.

why shouldn't the kernel be able to call into userspace safely and transfer ownership of a buffer? is that really so terrible to consider?

Well, if that's your biggest itch, be my guest: implement a kernel patch, make it public, convince people to use it, and if it develops a large user community, maybe Linus will pick it up and it will become a standard part of the kernel.

If nobody is willing to put in the effort, evidently the feature isn't needed.

Re:wrong (1)

Endymion (12816) | more than 4 years ago | (#27934611)

No, UNIX has these interfaces because they get the job done.

I think I'm going to have to add to my list of RFC1924 issues with this proposal...

"(1) It Has To Work."

I've never seen the socket interface to be rate limiting in anything I care about.

This whole topic stinks of a really bad case of Premature Optimization.

Re:wrong (0)

Anonymous Coward | more than 4 years ago | (#27935801)

why shouldn't the kernel be able to call into userspace safely and transfer ownership of a buffer? is that really so terrible to consider?

Rootkit much?

Re:wrong (4, Insightful)

jipn4 (1367823) | more than 4 years ago | (#27934207)

the most flexible and efficient thing would be to have memory for receive frames allocated at the bottom of the stack, and use callbacks all the way up.

Sure, in the same way that the "most flexible and efficient thing" would be to write inassembly language and turn off the MMU. But UNIX is not trying to do the most flexible and efficient thing, it's trying to be a reasonable tradeoff between simplicity, safety, and efficiency. And that means that efficiency only gets optimized to the point where it stops being a limiting factor for most programs.

Re:wrong (1)

convolvatron (176505) | more than 4 years ago | (#27934379)

but in this case we have structural flaws, which as you point out have some workarounds..some of which have their own problems. it seems reasonable to think about other approaches. i'm not going to buy into the tablets brought down from the berkeley hills.

gnn isn't really advocating throwing out sockets, you'll have to blame chellechelle for the inflammatory headline. queue is exactly that, a forum for discussing practice, and not a very deep one at that.

go ahead and live with your select and poll variants, they really aren't that bad. but i dont think they are the best that can be imagined.

Re:wrong (2, Insightful)

jipn4 (1367823) | more than 4 years ago | (#27934429)

but in this case we have structural flaws

Not conforming to someone's pipe dream of kernel design is not a flaw. It's a flaw only if it demonstrably causes problems.

i'm not going to buy into the tablets brought down from the berkeley hills.

That's why they make all kinds. You're free to use Windows Vista; those people spend billions correcting supposed "structural flaws". Don't spoil UNIX or Linux for the rest of us. We like its "structural flaws" the way they are.

Re:wrong (4, Interesting)

Darinbob (1142669) | more than 4 years ago | (#27934351)

But socket-like interfaces exist on systems without any user kernel interface. Ie, embedded systems. Many of those have implementations that do a good job of avoiding extra data copying, and yet still have an API that resembles sockets. I wonder if people are confusing the general idea of "sockets" with the specific "Berkeley Sockets" implementation and specification?

Structured Stream Transport (5, Informative)

ace123 (758107) | more than 4 years ago | (#27934027)

BSD sockets have a limitation of only a single stream at a time (for example, if you are loading a website over HTTP and you get stuck loading a huge image, you have no choice but to open up another socket connection or else wait). They are also stuck around the paradigm of only supporting byte streams, which means that users are always forced to write the same code over and over to create packet headers or delimited messages.

I would highly recommend checking out Structured Stream Transport [mit.edu]. I'm not from MIT and I wasn't entirely satisfied with their sample implementation, but the paper is really insightful and explains how you can develop basically a smarter version of TCP that is both more efficient and also more flexible. And I'm sure there are other systems being developed with similar ideas in mind.

We definitely need to keep bsd sockets, if not just because I'm a regular user of netcat :-p, and also because they are what allow the creation of more advanced protocols, but I don't think most applications should still be using such low-level protocols today.

Re:Structured Stream Transport (5, Informative)

Anonymous Coward | more than 4 years ago | (#27934185)

TCP != BSD Sockets

No matter how much abstraction you pile on top to open multiple streams, automatically add headers, communicate a fix message size to avoid in-band delimiters, etc., you'll still have to send all those messages over linear octet streams when using TCP.

Now you could choose not to use TCP -- UDP lets you send non-linear messages of arbitrary size without delimiters. And there may be other newer, better options available as well. But you can do both TCP and UDP (as well other other comm types) using the same sockets API.

Re:Structured Stream Transport (5, Informative)

Anonymous Coward | more than 4 years ago | (#27934255)

BSD sockets have a limitation of only a single stream at a time (for example, if you are loading a website over HTTP and you get stuck loading a huge image, you have no choice but to open up another socket connection or else wait).

No it doesn't. This is a limitation of TCP. You could just as easily use a different protocol (e.g., SCTP) with sockets.

Re:Structured Stream Transport (-1, Redundant)

Anonymous Coward | more than 4 years ago | (#27934325)

Mod parent up.

Re:Structured Stream Transport (1)

ace123 (758107) | more than 4 years ago | (#27934521)

I honestly had never heard of SCTP before, and I'm surprised that it is not used more widely since it has been around since 2000. It looks to be more complicated than what I was talking about since it covers more issues (talking to multiple hosts). Do you happen to know of any uses of this protocol in real applications?

BSD Sockets themselves are very flexible, I suppose I was complaining about the read/write semantics in stream sockets. Either way, it is possible to layer protocols even at the application level so it's not a big deal. Sadly I didn't get a chance to read the article before acm.org died.

Re:Structured Stream Transport (3, Insightful)

amorsen (7485) | more than 4 years ago | (#27934911)

I honestly had never heard of SCTP before, and I'm surprised that it is not used more widely since it has been around since 2000.

Firewalls don't support it. Consumer routers can't do NAT on it. New protocols on the Internet are fairly unlikely to have a chance.

Re:Structured Stream Transport (4, Insightful)

serviscope_minor (664417) | more than 4 years ago | (#27934501)

It is said that those who do not understand history are doomed to repeat it...

They are also stuck around the paradigm of only supporting byte streams, which means that users are always forced to write the same code over and over to create packet headers or delimited messages.

Byte streams is one of the UNIX fundamantals. Before UNIX, there were many systems which provided wide varieties of structured IO. This turned out to be a real pain and one of the UNIX innovations was simply to scrap it.

Ans today, most applications don't use low level sockets: they cal a library which does it for them. Moving the library in to the kernel is not a good idea.

Re:Structured Stream Transport (2, Interesting)

ace123 (758107) | more than 4 years ago | (#27934695)

I definitely agree with you. In fact byte streams being a fundamental part of POSIX is one thing I love and make use of every day, for example piping output between programs/sockets. My post was not very clear, but I was trying to say that users developing application protocols should not be using BSD sockets directly any more--people usually write or use libraries for that sort of thing.

As far as new protocols go, you can build basically anything using UDP (and UDP is far less likely to be firewalled than any custom IP-level protocol you make up). I think such a protocol could only ever be practically implemented user-space library anyway

I would be curious what the article thinks is so fundamentally wrong with the sockets paradigm.

Re:Structured Stream Transport (1)

Kjella (173770) | more than 4 years ago | (#27934985)

It is said that those who do not understand history are doomed to repeat it... (...) Byte streams is one of the UNIX fundamantals. Before UNIX, there were many systems which provided wide varieties of structured IO. This turned out to be a real pain and one of the UNIX innovations was simply to scrap it.

Ah, the "It's no longer our problem, thus the problem is solved" approach. While maybe it shouldn't be in the kernel, there's some things there should be only one of and basic messaging/IPC is one of them - looking at the wikipedia page there's more than two dozen listed and probably doesn't include the ancient pre-UNIX ways. It looks like finally the open source world is starting to settle on D-Bus as the core backend (Gnome, KDE and Win/Mac support) but that it's taken 40 years to get there exactly because UNIX/POSIX just left it at byte streams.

Maybe I'm thinking more in terms of a development platform than an OS platform, but just giving you the bare metal and "the rest can be done by libraries" is not my idea of a good solution. C++ is the worst example of a DIY library project, personally I prefer using C++/Qt but if I didn't I'd probably go with Java or C#. You really shouldn't have to go outside the standard library (for me adopting in the Qt library) to get what I'd consider programming "primitives", for advanced definitions of primitive. Libraries should be more about taking primitives and creating "modules" that you can drop in to get specific bits of functionality, not just e.g. boost to get a decent threading.

But then, I never was one of the C++ developers that love to fumble about with clever memory tricks and pointer magic. I'm much more concerned with getting the high-level right and not optimizing some algorithm way down there.

If you like structured IO (0)

Anonymous Coward | more than 4 years ago | (#27935377)

Go and use z/OS and it's MVS filesystem.

AAAAAARRRRGGGHHHHH!!!!!

Couldn't this be like a flag, rather than new API? (4, Interesting)

tjstork (137384) | more than 4 years ago | (#27934029)

he recently developed SCTP (Stream Control Transport Protocol)4 incorporates support for multihoming at the protocol level, but it is impossible to export this support through the sockets API

The word that bugs me there, is "impossible". The question is, why? If you have to do something with sockets under the hood, then so be it, but it would seem to me that you could just add a few more fields to socket address to take into account multiple homes.

We've already had alternative APIs to sockets and for quite some time. sockets won. There were named pipes, ipx/spx, and the seemingly stupid idea of treating a network resource as a file has trumped every time.

Re:Couldn't this be like a flag, rather than new A (4, Interesting)

phantomfive (622387) | more than 4 years ago | (#27934483)

The word that bugs me there, is "impossible". The question is, why? If you have to do something with sockets under the hood, then so be it, but it would seem to me that you could just add a few more fields to socket address to take into account multiple homes.

Especially since SCTP actually does use the sockets API. You have to use recvmsg() instead of recv() if you want to do multi-homing, but in using SCTP I was actually impressed by how flexible the BSD socket API actually is. I can't say I particularly like it, and everyone who uses it ends up writing a wrapper around most of the send and recv calls, but flexibility is definitely it's strong point. If we ever do get routing by carrier pigeon, the BSD socket API will be able to adapt to it.

Re:Couldn't this be like a flag, rather than new A (0)

Anonymous Coward | more than 4 years ago | (#27935585)

but flexibility is definitely it's strong point.
its*
sorry!

SCTP an interesting example (5, Interesting)

isj (453011) | more than 4 years ago | (#27934037)

I am developing SCTP applications and has contributed to the linux implementation, and I think that one of the advantages of the socket API is that it is usable with select()/ and poll(), ie. it is file descriptors you can pass around.

But for SCTP there are things that don't fit nicely into the socket API, especially when using one-to-many socket types. For instance for retrieving options for an association you have to piggyback data in a getsockopt() call by using the output buffer also for input. It works, but it is not nice. Also, for sending/receiving messages you have to use sendmsg/recvmsg with all the features including control data, and the ugly control data parsing.

Re:SCTP an interesting example (3, Informative)

QuoteMstr (55051) | more than 4 years ago | (#27934383)

It works, but it is not nice.

So use a wrapper, like sctp_send [die.net] from libsctp. There's no reason the kernel proper has to export these interfaces.

Re:SCTP an interesting example (1)

phantomfive (622387) | more than 4 years ago | (#27934837)

Just FYI: about a year ago I was writing a program with SCTP, and I kept getting kernel panics in the SCTP layer. It was annoying. YMMV.

Also, select() and poll() are both inefficient. I suggest you use epoll(). Once you get the hang of it, I think you will like the interface better as well.

Hmm... (3, Interesting)

fozzy1015 (264592) | more than 4 years ago | (#27934163)

In my experience the way the socket API can slow down a processor is having to monitor many thousands of socket descriptors using select() or poll(), like in a web server. For Linux epoll() was created for this scenario.

STREAMS? (2, Informative)

$lashdot (472358) | more than 4 years ago | (#27934169)

Macs used STREAMS from system 7.5.2 onwards. Was kind of sad to see that go away with the switch to OS X.

Absolutely! (0)

Anonymous Coward | more than 4 years ago | (#27934205)

I've been complaining about them for years. Maybe a generation ago, they were useful, but I think most people wear them out of tradition. The first thing I do when I get home is take off my shoes and socks. Also, I hate pants. ... oh! "Sockets" ... they're fine or whatever.

Old school socket set.... (-1, Offtopic)

rts008 (812749) | more than 4 years ago | (#27934315)

I have a socket set that covers SAE, and Metric stuff. I can work on any vehicle since 1960.
Is there a problem here?

Oh. it's just another 'change of language' that these damn punk kids insist on because they don't have the spine/balls to develop new concepts, and have to re-name the old to feel relevant?

Yeah, :
PC==Personal Computer
IP==Internet Protocol
Call me a Luddite all you want, but it does not change the fact you can't be original, and are 'climbing on the backs/shoulders of Giants' to achieve some nebulous 'marketshare'/Imaginary Property metric.

Change the terms and delude yourselves to your heart's content, it does not change the underlying facts, no matter how hard you protest.

Keep insisting that lose==loose, their==they're==there, a==an, and a multitude of other mistakes are 'okay' because it's 'on the internet' mentality.
Stupidity rules now!!!
Equality of the masses, never-mind most of you can extract your IQ from your shoe size!...We only have to be 'politically correct', not really correct...Just don't piss on the white-washed fence, and all will be okay.

If you want to go there:
"English, motherfucker, do you understand English?...say 'What' again, motherfucker!" *gunshot!!* (sorry if the concept is 'above' you)
Hint: watch 'Pulp Fiction' for the modern no-fault explanation.

Re:Old school socket set.... (0)

Anonymous Coward | more than 4 years ago | (#27934529)

Language evolves, get over it. Often the best name for some abstract concept is a metaphor using some concrete object. Your computer is full of strings and threads too! Call the dictionary police!

Oh, and that language your so possesive of? Yeah, hasn't been around very long (and the King would like to have a work with you about what you've done to His English).

Re:Old school socket set.... (0)

Anonymous Coward | more than 4 years ago | (#27934721)

Oh, and that language your so possesive of? Yeah, hasn't been around very long (and the King would like to have a work with you about what you've done to His English).

Case in point.

Re:Old school socket set.... (0)

Anonymous Coward | more than 4 years ago | (#27934753)

BSD sockets are also an 'old school set'. This might not be the site for you.

Re:Old school socket set.... (0)

Anonymous Coward | more than 4 years ago | (#27935241)

what the hell is wrong with you ?

IOCP (1)

kiss7 (1501315) | more than 4 years ago | (#27934341)

IOCP is perfect for both high bandwidth and low latency. ...you have to use the "ugly" windows os for it :)

It really seems to me... (2, Insightful)

Secret Rabbit (914973) | more than 4 years ago | (#27934353)

...that most of the things that this guy is talking about would be better implemented below the sockets API. As in, how the OS handles things. Making things transparent is a good thing.

I'll also point out that having a fail over interface so that the client doesn't lose the connection has already been done in OpenBSD's pf called CARP. It is a free alternative to VRRP and HSRP. In other words, this doesn't have to be implemented in the API when another avenue already exists that does it.

Yes Mine are good (3, Funny)

ben2umbc (1090351) | more than 4 years ago | (#27934425)

My socks are fine for now. When they do run their course I go to walmart and get new socks its $5 for 6 pair!

User level networking and the last copy (4, Interesting)

wdebruij (239038) | more than 4 years ago | (#27934427)

This is hardly news and partly mistaken.

The statement that sockets limit throughput by copying between kernel and application processes is a bit simplistic. The copy of Rx data to an application usually primes the cache. If data isn't touched and loaded into the cache at this point, it will have to be loaded shortly, anyway. Granted, for Tx this trick does not hold.

Second, the interface is not the implementation. Just because sockets are traditionally implemented as system calls does not state that they have to. User level networking is a well known alternative to OS services for high-bandwidth and low-latency communication (e.g., U-net [cornell.edu] developed around '96). I know, because I myself built a network stack [netstreamline.org] with large shared buffers [netstreamline.org] that implements the socket API through local function calls (blatant plug, but on topic. The implementation is still shoddy, but good enough for UDP benchmarking).

User level networking can also offers low latency. My implementation doesn't, but U-net does.

This leaves the third point of the article, on multihoming. As sockets abstract away IP addresses and network interfaces, I don't see why they cannot support multihoming behind the socket interface. Note that IP addresses do not have to mapped 1:1 onto NICs. Operating systems generally support load-balancing or fail-over behind the interface through virtual interfaces (in IRIX) or some other means (Netfilter in [sgi.com]Linux [tetro.net]).

Not need to replace sockets just yet.

Should stop drinking... (1, Funny)

Anonymous Coward | more than 4 years ago | (#27934441)

Have Rockets Run Their Course?

Bad (0, Redundant)

poor_boi (548340) | more than 4 years ago | (#27934633)

This is a bad article and a bad thread and you all should feel bad for posting it and taking it seriously and, finally, for reading this -- my post/

Re:Bad (0)

Anonymous Coward | more than 4 years ago | (#27934743)

just... wow. Seriously, get over it, it's just a slashdot article

Alternatives (1)

RAMMS+EIN (578166) | more than 4 years ago | (#27934661)

I couldn't get to the article, but if they think Berkeley sockets are obsolete, I'd like to see what alternative they offer, why they think these alternatives are better, and what the pitfalls of the alternatives are.

It's not sockets, its bind() (5, Interesting)

argent (18001) | more than 4 years ago | (#27934907)

The socket API... or rather the UNIX file descriptor API... has been extended many times. Sockets are already one such extension, and there's no reason you couldn't do something like mmap() a socket to map the buffers into user space directly. Heck, udp sockets already diverge from the read/write paradigm.

The problem with sockets is at a higher level. They're not mapped into the file system name space. You should be able to open a socket by calling open() on something like "/dev/tcp/address-or-name/port-or-name" and completely hide the details of gethostbyname(), bind(), and so on from the application layer. If they'd done that we'd already be using IPv6 for everything because applications wouldn't have to know about the details of addresses because they'd just be arbitrary strings like file names already are.

Plan 9 (2, Informative)

Anonymous Coward | more than 4 years ago | (#27935401)

Already does that.

Low latency? (1)

sphealey (2855) | more than 4 years ago | (#27935301)

"...high bandwidth, low latency..."? Low latency? Is the author working on some alternative universe Internet with low latency, rather than the high, increasing, and highly variable latency of the Internet here in this universe/on this planet? Or perhaps he has a telco that isn't continuously raising the price of T1s and T3s to force him onto high-latency IP connectivity "solutions"?

sPh

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...