Hatrack River Forum: The geek thread

This is topic The geek thread in forum Books, Films, Food and Culture at Hatrack River Forum.

To visit this topic, use this URL:
http://www.hatrack.com/ubb/main/ultimatebb.php?ubb=get_topic;f=2;t=025625

Posted by Hobbes (Member # 433) on :

With all this bragging and discussing of who is and is not a geek, I figured us geeks should have a place to discuss geeky things, instead of just arguing about geek hood.

For my part, I’ve come to like my job a lot more lately. I started off programming in C, and even a little web-development, and server work for a while. Now, however, because of some complications with SOAP I’m programming in Java. I’m not really a huge fan of Java, but for the first time I get to work in an actual development environment (I couldn’t get my C++ compilers installed on these computers and I’ve been stuck using text editors). Eclipse is a great environment, I like VS .NET better, but hey, Eclipse is free. Anyways, I’m now able to use an OO language to program in as well as a fully integrated development environment so I’m happy. Of course I’m supposed to implement a web-service based only on code I’ve written using Java, SOAP and XML, and I don’t really know those last two languages. But I am doing it on an Apache server so that’s good (on a non-standard release of Red-Hat that has already caused me to need to re-compile an entire server).

Hobbes [Smile]

Posted by Erik Slaine (Member # 5583) on :

*flees*

Posted by saxon75 (Member # 4589) on :

Argh! The Geeks Club thread fell off the end!

Posted by Snarky (Member # 4406) on :

*moment of silence for the now-defunt geeks thread*

Posted by Teshi (Member # 5024) on :

<---- prefers term 'gnork' to 'geek'

Posted by HollowEarth (Member # 2586) on :

^
|
|
is a sign of a true geek.

Posted by fugu13 (Member # 2859) on :

Use Apache Axis. It handles all the SOAP details for you on simpler web services, and makes it easier to deal with them yourself on more complex web services.

Posted by fugu13 (Member # 2859) on :

Oh, and feel free to use me as a resource. I'm well acquainted with SOAP and XML [Smile]

.

[ July 01, 2004, 09:58 PM: Message edited by: fugu13 ]

Posted by Eruve Nandiriel (Member # 5677) on :

I get to learn some C programming this year! [Big Grin]

I used to use some Q-Basic, until I got really disgusted when I didn't back up the program often enough, and couldn't locate the problem in my game... [Grumble]

Posted by Bekenn (Member # 6602) on :

I actually started way back with GWBasic, shortly after King's Quest 1 (EGA) taught me how to type on my PC Jr.

Posted by Hobbes (Member # 433) on :

You're on Fugu! I'll tell you staright out what I need to do, I will have to transfer two files from the client computer to the server, and one file back. Of the two files transfered, one of them can be marked up in xml (though it certainly doesn't have to be), and the other one is simply to large to make that practical (actually, it'll probably be to large to make transfering it in any format impractical, but it's not my call, my employer wants the web service). Once the file reachs the server I have to run a program (sounds like Java is the easiest language to do this in) that will take the smaller file, and use that to process the larger one. Then, it will send another file (too large for xml again) back.

It looks like MIME may be a possibility for this. But about all I've discovered so far is that xml is a general mark up language that uses user-defined tags and that SOAP uses XML and http. Suggestions?

Hobbes [Smile]

Posted by Hobbes (Member # 433) on :

<?xml version = "1.0" encoding = :UTF-8"?>
<documnet>
<preamble>In other words</preamble>
<main point = "this is about all I know"/>
<secondary>And I know more about XML then about SOAP</secondary>
<final>A LOT more about XML</final>
</document>

Hobbes [Smile]

Posted by fugu13 (Member # 2859) on :

*chuckle*

Okay, define a method that takes two strings (the files) as arguments, then returns a string (the return file).

Then go here: http://ws.apache.org/axis/ and read the basic tutorial on creating a SOAP webservice ( http://ws.apache.org/axis/java/user-guide.html , read the Publishing Web Services with Axis section in particular).

Create your webservice using the method you defined. Since there'll be heavy processing per request, the JWS method of doing so should be fine. At least for development, and while you're still learning.

Make calls to that webservice as described here: http://ws.apache.org/axis/java/client-side-axis.html

Actually, you'll find it easier to make calls (assuming its from a client you're also writing, and can use Java for) if you use the more complicated method of deployment described on the first page. That automagically generates libraries that can do the client calls pretty much transparently.

As for XML, time to start reading. There are two basic methods of parsing XML -- DOM and SAX. In the DOM, an entire XML document is read into memory and made into an object based tree structure. In SAX, you create event handlers which the parser triggers as it goes through and encounters tags. Sounds like you'll probably want to parse the smaller file into the DOM (assuming its "small enough"), then use that to perform some custom parsing on the larger, non XML file.

Posted by Dan_raven (Member # 3383) on :

hmph.

Its all Geek to me.

(someone had to say it.)

Posted by Hobbes (Member # 433) on :

OK Fugu, I have to say, I've never really done any internet applications, server side or client side, so let's start off from the begining, and on the client side. I wrote up a little html file with a form in it that I would think looks like a typical CGI submit form. Obviously it's not going to be using SOAP, so how would I have this work?

http://wahra.babylonia.flatirons.org/Form.html

Hobbes [Smile]

Posted by HollowEarth (Member # 2586) on :

Sounds to me like you would be best off hitting the library or bookstore.

Posted by Hobbes (Member # 433) on :

I have, but unfortunatly nothing really hits on the client-side aspect of it, and the only book I could find on SOAP only talked about Server side from the perspective of someone who clearly knows all the basics of XML and CGI and has no interest in discussing them. I'm trying to frantically learn XML but it does take a while, especially since this isn't the only thing I have to do.

Hobbes [Smile]

Posted by fugu13 (Member # 2859) on :

okay, basics of the web. The web is a system whereby one requests documents using messages. The messages sent are sometimes complex enough to be called documents in their own right.

in SOAP (the Simple Object Access Protocol), the message which is sent is a request to access an object. It is sent in the form of an XML document, the details of which don't need to concern you as you should never create one by hand. There are abundant libraries for dealing with every situation except the most arcane. Arcane in the sense of a guy with a doctorate in CS and Art figuring out how to have a series of SOAP calls form an ASCII art animation. The XML document encodes the method call, preserving types (well, many basic types). One can provide adapters for other types, though this must be understood on the server and client ends. Complex types are allowed that are formed via composites and such of simpler types.

What is returned is an XML document, much like the message that was sent, which contains encoded within it the result, also with type.

SOAP is an RPC mechanism. Remote Procedure Call. Its pretty self explanatory -- when you make a SOAP call, you are calling a procedure at a remote location.

Another common RPC mechanism is XML-RPC, which is lighter weight, but does not have the capabilities (in particular, for out of band (that is, not handled by the method call itself) security) that a SOAP call has.

A web form is not what you want, I think. If you want this to be programmatic on both ends, it is fitting a square peg into a round hole. Web forms are for human interaction, and for reasons of backwards compatibility and legacy systems and such are quite limited. In particular they lack type, which is inordinately useful in any complex programmatic endeavor.

There are several things to remember wrt the web. First, all communication is heavy. This is why good libraries do not act completely transparently -- that invites inefficient use by excessive method calls. (One particularly useful concept on the web is idempotent behavior -- a given method call changes the object to a certain state, performing all required modifications and method calls regardless of the state the object started in. This prevents the excessive use of method calls.)

Another thing about the web is that it is unreliable. Many things can go wrong, preventing the return of a result. Using a library to handle SOAP ensures that these errors are handle-able, or if not handled do not allow the program to proceed (uncaught exception).

Further questions?

Posted by Hobbes (Member # 433) on :

So you're saying that do really use SOAP in meaningful way, the user will have to download a program? The reason my employer wanted a web service was so that they wouldn't have to do that. If I tell him that this can't be done, at least not with SOAP, I think he'll take my word for it and either tell me to write something that will work on multiple systems, or use another way of doing this. If everything I said is right, and he goes with option 2, any suggestions?

Hobbes [Smile]

Posted by fugu13 (Member # 2859) on :

What is this web service (whatever style of calling mechanism is used) going to be doing?

Posted by Hobbes (Member # 433) on :

For the most part it takes a big fat file in some random format (and then a metadata file to describe, one that I would prefer be created through forms if it's going to be a web service) and converts it from the general format into a specific data format. And of course sends the file back for the user.

Hobbes [Smile]

Posted by Hobbes (Member # 433) on :

To be honest, I would really prefer to make this a download only program, and it makes a heck of a lot more sense, especially since some of these files are many megabytes, sometimes even gigabytes, but my employer really seemed to be instent on the web-service.

Hobbes [Smile]

Posted by saxon75 (Member # 4589) on :

OK, so since this is turning into an "Ask the Geek" thread, I'll throw one in:

I've been developing a web app that uses the MVC framework, and I am getting to the point where I need to decide how to handle storage. In the application, I use a custom object to store the data, and I write the entire object to a session variable in order to preserve data from page to page. I am planning on using a MySQL database to store the information between sessions. Is it more extensible to extract the data from the object and store each datum in a different field, or to simply store the entire object as a binary blob? Is extensibility the only concern here? Is there some other reason I might want to do one instead of the other?

Posted by Hobbes (Member # 433) on :

Are you worried about storage size, or just ease of access?

Hobbes [Smile]

Posted by saxon75 (Member # 4589) on :

I'm not too worried about either, really.

Posted by Hobbes (Member # 433) on :

My vote would be fields then. It makes it easier to acess the data in a non-cryptic way. In other words, when you do a blob and you know what every single item is, that'll work, but if you want to add or delete something from the blob, it can be a big pain, and a lot of trying to remember what came when. Also, if you go away from the code for a while and then come back, well the blow will take way longer to re-learn.

But that's just my two cents.

Hobbes [Smile]

Posted by fugu13 (Member # 2859) on :

saxon -- you want some sort of Object-Relational bridge solution, like Hibernate ( http://www.hibernate.org/ ) provides (for Java). That will handle most of the details for you.

Hobbes -- gigabytes? yeah, not suited for anyone not on Internet2. You could just do it as a CGI program. You'd create a form, where a user selected the two files, that was then submitted to a CGI program. The CGI program would take those two files, parsing them out of the POST variables (you'd use a CGI library to do this), process them, and then return the result as the appropriate type of document. Doing extensive input checking to prevent hacks, of course. You'd probably also want to provide it as a SOAP service (give each user a generated key, like amazon, and cut them off after a certain number of uses per day or somesuch), for people who want to access the parser programmatically.

Posted by Hobbes (Member # 433) on :

Though I have to admit that one of my favorite ways of saving a class to file in C++ (admittedly a very different situation than yours) is this:

code:

class Object:
...
};

int main (int argc, char **argv) {
Object ToBeSaved;
Object ToBeWritten;
...
ofstream out;
out.open ("filename.ext", ios::binary);
out.write ((char *)(&ToBeSaved), sizeof (Object));
out.close ();

...

ifstream in;
in.open ("filename.ext", ios::binary);
in.read ((char *)(&ToBeWritten), sizeof (Object));
in.close ();

...

return 0;
}

Hobbes

Posted by Hobbes (Member # 433) on :

Well I wouldn't worry about overuse, this is for a geophysics project and converting to the very popular (really) scientific file format called "NetCDF". We've got a really nice, really fast server here, and there's no way the demand could outstrip the supply.

Thanks for all your help! No I just have to wait for my Employer to get back from vacation and tell him to scrap the SOAP transfer idea. [Big Grin]

Hobbes

[ July 02, 2004, 02:38 PM: Message edited by: Hobbes ]

Posted by saxon75 (Member # 4589) on :

::head asplodes::

fugu, that website was a little over my head. It looks like that would do what I want (supposing I could find a PHP library that does the same thing), but I'm kind of lost as to how that would be better than just doing it myself. I mean, you have to write a mapping, so if I were to add, delete, or change a member variable of the object I'm trying to store, I would have to make two changes: one in the class declaration and one in the mapping. If I do it myself, I still only have to make two changes: one in the class declaration and one in the function extracting and storing the data.

Posted by fugu13 (Member # 2859) on :

You only have one function doing both the extracting and the storing [Wink]

?

Also, you're neglecting one very important problem with rolling your own solution: non-elementary fields. What if one (or more) of the fields of the object you wish to store contains an object? Suddenly you have to add recursive object building routines with callbacks into your method. A lot more complicated.

Of course, if such a situation will never arise, fields are okay-ish. They're just inflexible.

Posted by fugu13 (Member # 2859) on :

Oh, and as for why not to store stuff in binary: binary formats can change with compiler, or even with version of compiler, depending on language and such. PHP is particularly nonstandardized in implementation, storing binary objects will probably not be portable between PHP versions.

The C++ standard is better, but still allows for significant flexibility in object binary format. Incompatible flexibility.

One is usually pretty safe with Java, but one also has an excellent object relational bridge in Java, making such considerations unnecessary.

Posted by saxon75 (Member # 4589) on :

Remind me never to show you any of my source. [Wink]

Posted by fugu13 (Member # 2859) on :

[Big Grin]

Don't worry, I'd be gentle . . . really!

Posted by saxon75 (Member # 4589) on :

Actually, I'm sure you would be, but I've finally gotten this project into a place where it's fairly easy to work with and I'm feeling good about finally imposing some order and organization on my hobby work. Despite all that, I'm positive that there are all kinds of things I'm doing wrong, both in terms of function and organization, and I don't know if my fragile programming ego could handle it if they were all pointed out.

Posted by saxon75 (Member # 4589) on :

Another question:

Is Python really a good replacement for PHP? That is, my understanding of Python on the web is that you can use it to write CGI scripts. But isn't CGI quite a bit different from dynamic web pages?

Posted by fugu13 (Member # 2859) on :

No.

One can write python pretty much as one writes php. There's even a mod_python.

Oh, and its a superior replacement for PHP.

[ July 02, 2004, 05:03 PM: Message edited by: fugu13 ]

Posted by saxon75 (Member # 4589) on :

Hmmm... I thought I remembered reading something somewhere that said that there were things you couldn't do in CGI that you can do with things like ASP and PHP. Maybe I'm thinking of applets. I forget.

Posted by fugu13 (Member # 2859) on :

Nope. Now, some apache functions aren't available unless you're running mod_whatever on it, but other than that there's no particular difference (oh, beyond which directory they're run in, memory efficiency, that sort of thing, but python has such efficiencies in mod_python just like php does in mod_php; there aren't any functionality limitations).

Posted by saxon75 (Member # 4589) on :

Right on. I'll keep that in mind if I should ever get my own server (or run across a hosting service that loads mod_python). [Smile]

Posted by Jenny Gardener (Member # 903) on :

Egads! This thread is making me breathless!

*Jenny pulls on a very tight, cut-off t-shirt emblazoned with the message "I Love Geeks"*

I want to start the Geek fan club.

Posted by saxon75 (Member # 4589) on :

Don't let Juliette see you in that shirt while I'm around. [Razz]

Posted by Jenny Gardener (Member # 903) on :

Good thing I've got my own geek. But all this computer talk is making me HOT!

You all are lucky I'm married, else you'd be stalked. I do recommend geeks to my very pretty 16-year-old sister. She's tall, buxom, athletic, and fun. She's not really a geek, though.

Geeks are HOT! [Kiss]