15,000 records inserted into the database == 1 hour. Now, there are a couple of gotcha's here... The stored procedure being used to do inserts wasn't meant for batch inserting. All of the indexes were on, including a text context index on a CLOB field. Lets not forget about the triggers that are used to keep history. Just to ensure everything was inserted appropriately, I was using our Java classes that interface with the database, which also were not meant for batch inserts. Given all of that... I still don't think it should have taken 1 hour to insert 15,000 records.
So your next question is... Hardware? Well, it isn't running on a 486... The database isn't even on Intel, it is on a Sparc. Well... Sparcs can be slow too... But not 880's with 4GB of RAM and 2 CPU's.
At this pace, the batch should be finished by Sunday night at 9pm. That's if it doesn't die. I Just want to know how well it will perform on Monday morning when we try to do a search against that table... with its 750,000 extra rows :)
But... now that I think about it... I bet most of that time is spent connecting to the database over and over again.
So I took my code for testing out Jaxor and actually implemented the appropriate JSP's and Actions to go along to take my theories for a spin around the webapp. It all works. My Actions are a bit ugly because they are very specific to the Use Case that I am using for testing, so they will need refactoring soon. I have functionality to add and view Entities. The one problem that I bumped into was with JSTL not approving of the Entity specific List being returned. I noticed a toList() method in the Entity's List object, so I added a getList() that returns toList() so that JSTL will know what to do with it all. Of course if I was using Velocity, I may not have that problem.
I didn't have any problems with the lazy loading... I put JaxorContext's start and end in the BaseAction, but then my list only has 3 entries, so it may not be affecting me right now.
I think it would be nifty if IDEA would track how much time you are putting into a project. It would be really cool if the tracker would also keep track of the files that you edited as well. I keep forgetting to write down what time I start writing code and what time I stop.
I installed the SimpleUML and the Sequence Diagram plugins. Very nice additions that I will be taking to work with me on Monday. I guess I should probably leave Tetris at home though :)
I have upgraded my app to use the Jaxor Entities in my Struts ActionForms instead of an ADM (Anemic Domain Model) like I said I would do in a previous post. I didn't know if it would work or not because I am still learning Jaxor. Well, Jaxor pulled through again and provided the necessary methods for me to pull this off. It was very exciting because I had tests that worked. So I actually got to practice TDD. Granted I am not doing Test First, I still can't get my brain wrapped around that, but I think I see it coming in the near term. Keeping that green bar hopping across my JUnit window was lots of fun.
Anyway, back to Jaxor and Struts. My ActionForm is very complex with 5 private member variables<sarcasm> 4 of the 5 are Jaxor Entity Objects and the 5th is a boolean. I created a Business Delegate Object that the Action will call. At work, our Actions have started getting very complicated because they are storing the business logic. So, this time I am going to take my own advice and put a Business Delegate class out there to handle the details and allow my Action to play traffic cop just as he should.
The primary reason for refactoring my ActionForms to use Jaxor Entities was to get rid of the need to copy the properties from one object to another. Having to maintain two objects with the exact same properties was just... Dumb. While doing this, I found more improvements and significantly improved the readability of my code. Since Struts will be sending in new Base Entities that are not really new instances of the Entity (IE no JaxorContext) I came up with a really nifty method that takes an Entity, sets the JaxorContext, calls insert(), commit()'s and then returns the entity. A very sexy 4 lines of code that is currently used in 3 different places, but will end up being used often.
Thank you Lord for my beautiful kids who are wonderful and are growing up and becoming great little people. Thank You for my friends Cris, Ed, Matt and Nolan; each of whom has helped me grow in different ways. Thank You for my new employer, who saw the potential You gave me. Thank you Lord for my wife who always takes care of me. Thank you Lord for bringing these wonderful people into my life.
Woo hooo!! I am unemployed as of 12:30pm today. I have a job lined up and I start Monday, but it just seems fun to say that I am unemployed. This move is a small risk, much smaller than when I joined a dotcom back in 2000, but it is a risk just the same. My new company is tiny, I will be the sixth employee. I am very excited.
One of my friends absolutely hates JSTL. With a capital H. He believes that there is no reason to learn another language when he already knows Java... which works just fine in JSP's. I was tasked to add some functionality to one of his JSP's. Which humorously has scriptlets and JSTL mixed. Maybe confusingly would be a better description. He understands the fundamentals of JSTL and is a master at Java, but asking him to <c:set var=... just drives him up the wall.
I don't understand this at all... but I am starting to. <c:choose> takes a truckload more code to do a simple if("blah".equals(blah) || "something".equals(blah)) What is so bad about <%= car.getMake().getModel() %> Is it that much better than <c:out value="${car.make.model}"/> I doubt it. Which is one of the reasons I am going to start taking a serious look at Velocity and how well it integrates with Struts.
Mike really came through with a great tutorial on entity relationships. I was off and running building new tests and features in a matter of minutes. I was avoiding creating a mapper for my link table because I thought (for some weird reason not related to Jaxor) that Jaxor was going to handle the linkages for me. I think if I had given that a try I would have found the correct path, it just would have taken a while longer... I would have taken until Saturday, but I would have figured it out :)
My knowledge of Jaxor has gone from a dull spoon to a steak knife. Watch out folks, he's dangerous!
I would like to be a little more specific about what I learned yesterday... One of the doofus ideas that I have hung onto for so long was vanquished yesterday. At some point in time, I got it into my head that you should insert info into a db, then go and retrieve it so that you could get the id. Mike's suggestion of generating the PK up front has been suggested before, and I have dismissed it for various reason, none of which are very technically sound. This time, I gave it a try because I have learned (recently) that doing things the "right way" makes the rest of the work easier.
I am starting to understand that some of my biggest problems are my own misconceptions. Things that I have "learned" in the past may not be the "right way" and I may be robbing myself of the benefits presented by the new things I am learning today. Tomorrow, I will replace my puny DO's with monster Jaxor Entities and see where that road leads.
I think my new knife is already starting to make a differnce...
Jaxor has been a real champ today. I learned a couple of things about refactoring and about returning objects instead of id's. This is all a little bit new to me, just because I am a rook :) Most of this learning was accomplished while unit testing my code. I have one method in each of my entities that does the same thing, so I am going to look at how to possibly refactor that code into one method for all of my entities.
I still haven't had time to figure out how to do many-to-many, but I am sure I will figure it out once I start looking into it. They are the next part of my application that I am building to help learn Jaxor.
This little bug was driving me bonkers. From time to time, the submit buttons on our web application would stop working, but only in IE. We couldn't figure it out. Then, one of our testers entered a bug about a note field not accepting more than 4000 characters. Starting with a clean slate, all submit buttons were working. Then I added a 18,000 character string (I got a little carried away with CTRL-V) and the submit buttons stopped working! I thought about it and deleted my 18,000 character paragraph and low and behold, the submit buttons are working again!
IE is being really smart here... and stupid at the same time :) IE knows that what you are trying to send through GET will not work, so it just refuses to send the information. If I change the form's method to POST IE has no problems sending in an 18,000 character string. I just wish IE would have let me know WHY it had disabled the submit functionality, instead of just refusing to execute.
I have been working on a little Jaxor project over the last couple of days. I must say that the implementation is very nice. It throws NPE's when you haven't made a connection, so trouble shooting that was a bit of a pain. It would have been nice if it had caught the NPE and thrown a PleaseMakeAConnectionBeforeTryingToCommitException instead :) Also, the docs are... not existent. I have finally come to the realization that good docs make or break an open source project. iBatis scores huge points for having the best docs for any of the persistence frameworks I have used.
Whoa! I guess the complaints about the weird libraries dialog made an impact because JetBrains brings out a brand new project configuration dialog that is seperated from the project properties.
I am interested in using #977, but there is a nifty bug that causes some freaky flashing on the screen while using it. Apparently it has already been reported as a bug, so I will stick with #963 for now. I suggest downloading #977 anyway just to check out the new libraries/classpath and sourcepath configuration dialog box.
Here is a great article about testing and what happens when a bug is found. What decisions have to be made when a bug is found.
Good persistence layers are hard to find. I was thinking about trying to implement on based on Fowler's PEAA Mapping Patterns. Apparently, others thought that was a good idea too because Jaxor has just been released into the wild. I am betting that Jaxor will be a very successful open source tool. Watch out Hibernate.
I agree that JSTL can be a bit "verbose", but the metaphors used by JSTL successfully convey their application. Useful metaphors are something that many smart people often overlook when talking about a framework’s proficiency. You may understand what a Result is because you wrote the code, but to a novice, Dispatch is a more readily apparent metaphor. I think that the best example for JSTL being verbose is the choose/when/otherwise tags that are used for logic. However, I am pretty sure I could give a novice developer a JSTL book and he would quickly understand how to use those tags. “CHOOSE one of these groups WHEN this is true or WHEN this is true, OTHERWISE do something else.”
Good metaphors are one of the reasons why Struts makes sense. Struts has Actions, they do things, and Forms, you fill them out, and it is all nicely tied together with a simple XML file. Simple.
Deriding JSTL for flexibility is short-sited. Claiming Struts is inferior because it only does web applications is nonsensical. From the Struts web site, “Welcome to Struts! The goal of this project is to provide an open source framework for building Java web applications.” I think it is pretty impressive that they are staying focused on one thing instead of trying to be everything to everyone.
Struts and JSTL are attractive because they are flexible. If you are a novice, you can easily get started and build a nice site. Said site won't win "code of the year" awards, but it will work. You don't have to know IoC, Command Pattern or whatever is vogue this month to make a Struts site. If you need a more robust system, then Struts will get you there, maybe not as elegantly, but you will get there (see Roller).
I believe Russ said it best, but I will summarize with, “Why build a web app when a JSTL page will do?”
I recently built a tool in JSTL. 1 file, 60 lines. If I built the same tool in Struts I would need to create a JSP, Action and Form and then I would also have to change the struts-config.xml. JSTL may be "so 1995" but there is serious power in those 60 lines all packaged nicely in one file. Use the right tool for the job. Struts and JSTL may not be very sexy, but they get the job done, with a low barrier to entry to boot.
My next assignment is going to require a bit of communication between multiple servers. We thought about going with a Web Services/SOAP implementation but I have kept JMS in the back of my head since I started learning about the project. One of the talks that I attended at NFJS was on JMS. It was a real stinker and I was very disappointed. During my morning read of Erik's site I found a nice little article on the players in the JMS arena. From that article I found this little overview of JMS. Which is exactly what I was looking for this weekend.
After reading the article I am now pretty sure that we will be using JMS instead of Web Services. How exactly? Well, I will need to do some more reading on the subject before I make that decision.
NFJS was so great this weekend. Jay Zimmerman runs a smooth show. It is a small symposium that runs with the professionalism of a giant conference. The quality of the speakers is one of the greatest reasons to attend.
I attended Erik's presentations on Velocity and Lucene. I am very much looking forward to his book on Lucene. Dave gave a great talk on Decoupling Patterns. Stuart had a couple of great presentations as well.
It was a great weekend!
After the day's session at NFJS here in the DC area, we all made our way to my friend Scott's house for some dinner and fun. I have a couple of pictures and I will try to get a couple more tomorrow during the day. I met a bunch of smart, fun people and had a great time. It is late, I am tired, so.... Good night.
Tomorrow is going to be so great. It will be my first trip to a NFJS. My friends have said that it is great. I am definately looking forward to Erik Hatcher's Velocity and Lucene talks. I am pretty upset about having to miss Stuart Halloway talking about Reflection. It was a hard choice to make, but I expect to start using Velocity on a regular basis in the very near future and I know others from my team will be in the Reflection talk.
Big shock. I was lagging behind on my downloads of EAP builds and when I actually install and start to use one, JetBrains releases a new build. :: sigh :: I am off to download build 963 now.
I am seriously considering taking the EAP builds to work for use. I really really really like the ability to split the view and edit two different files at the same time. With the debugger getting some new JSP support and the added support for WebLogic, this build is going to be a dream to use.
JetBrains continues to improve IDEA. Download build #957 today. I used the split panes tonight, that was really nice. Even better than having Notepad opened on my second monitor. Now I just need a bigger monitor :) Time to upgrade to the twenty-one inch...
SSH support for CVS is seemless. The CVS plugin is a champion in my book. I right clicked on a new folder (whose letters were brown letting me know that it had not been added yet) and chose Add... This popped up a dialog that allowed me to choose the type of files and exclude those that I did not want to be committed. Very slick.
Three cheers for Live Templates. Saved the day again, but this time in generating SQL. I created templates for creating tables, foreign keys, integer and varchar columns. Creating a schema has never been so easy.
But IDEA is not without its flaws. I am annoyed by the project taking a level all by itself in the project view. Unless they add support to have multiple projects open in the same window, this is a waste of valuable space. A minor fuss.
My previous post was prompted by a trip to my local CompUSA. I picked up a Tungsten C and checked out my blog. It looked horrible. Other items of note, a 17" Toshiba that wasn't a desktop replacement, it was a desk. It was no where near as nice as a 17" PowerBook. CompUSA is starting to cater to the hardcore hardware industry with fancy cables and cases. It seems that they are selling local brands too. One of the local computer stores had branded systems sitting on the CompUSA shelves. They are moving into the Home Entertainment space in a big way. BestBuy should be scared.
Over the last couple of months, I have really started getting interested in mobile technologies. New phones from Nokia and Seimens have really caught my eye, not to mention the new PDA's that are coming out like the Tungsten C. All of these mobile devices have the capability to browse the internet, but at a much lower resolution than a typical browser. Back in the day, during the Internet boom, web developers had to worry about 800x600 and 640x480. Everyone had a sigh of relief when 1024x768 started to become norm. It allowed for all kinds of new things to be done. Humorously, technology is forcing us to go in the exact OPPOSITE direction now. Now, we have to start developing for 320x320 and smaller.
I know the Internet has been available to cellphone users before, but it was crippled. WML has been out for a long time, but it was completely a niche market. You had to REALLY want to provide content for mobile users to go through the pain of delivering to mobile devices. Now, the full experience is available, pretty pictures and all. The demand for 320x320 web sites is going to explode in the very near future. I know my sites are not ready for it, are your's?