Comments on: Why not to use Excel for Data Gathering http://caseelse.net/2008/08/06/why-not-to-use-excel-for-data-gathering/ /*** Code's last stand ***/ Mon, 24 Oct 2011 08:14:51 +0000 hourly 1 http://wordpress.org/?v=3.3.2 By: ravenhall http://caseelse.net/2008/08/06/why-not-to-use-excel-for-data-gathering/comment-page-1/#comment-358 ravenhall Fri, 21 Oct 2011 15:07:47 +0000 http://caseelse.net/?p=36#comment-358 Python Crusader: You just show how little you know about Perl. Perl 6 is a new language - *not* the successor to Perl 5. Perl 5 is doing quite well and the latest stable is 5.14.2 released 2011-09-26. The bleeding edge is 5.15.4, released 2011-10-20 - so your argument is utterly null and void. The Perl modules for working with both Excel and virtually every type of database in existence are mature and solid. Perl is indeed an excellent choice for automating a data import from Excel to a database. Excel n00bs who learn a pivot table and think it's a nifty trick that they can use for everything might temporarily wow their management, but they and their machines quickly become overwhelmed by the sheer inefficiency. Their charts become more and more jumbled and meaningless as they grow in complexity, because they almost always know nothing about true data modelling. They only end up confirming their own incompetence, and a programmer /DBA inevitably has to bail them out. Python Crusader:

You just show how little you know about Perl. Perl 6 is a new language – *not* the successor to Perl 5. Perl 5 is doing quite well and the latest stable is 5.14.2 released 2011-09-26. The bleeding edge is 5.15.4, released 2011-10-20 – so your argument is utterly null and void.

The Perl modules for working with both Excel and virtually every type of database in existence are mature and solid. Perl is indeed an excellent choice for automating a data import from Excel to a database.

Excel n00bs who learn a pivot table and think it’s a nifty trick that they can use for everything might temporarily wow their management, but they and their machines quickly become overwhelmed by the sheer inefficiency. Their charts become more and more jumbled and meaningless as they grow in complexity, because they almost always know nothing about true data modelling. They only end up confirming their own incompetence, and a programmer /DBA inevitably has to bail them out.

]]> By: Mike http://caseelse.net/2008/08/06/why-not-to-use-excel-for-data-gathering/comment-page-1/#comment-345 Mike Tue, 11 May 2010 13:02:59 +0000 http://caseelse.net/?p=36#comment-345 @Ian Welsh: That is the problem. Most managers, once they get to the office/level they're at, seem to lost all sense of what is right and wrong - in terms of tools to do the job. @Ian Welsh: That is the problem. Most managers, once they get to the office/level they’re at, seem to lost all sense of what is right and wrong – in terms of tools to do the job.

]]> By: Johan L http://caseelse.net/2008/08/06/why-not-to-use-excel-for-data-gathering/comment-page-1/#comment-55 Johan L Sun, 10 Aug 2008 22:35:30 +0000 http://caseelse.net/?p=36#comment-55 Nice troll. <a href="http://blog.timbunce.org/2008/03/08/perl-myths/" rel="nofollow">Reality check</a> Nice troll.
Reality check

]]> By: Neil http://caseelse.net/2008/08/06/why-not-to-use-excel-for-data-gathering/comment-page-1/#comment-54 Neil Sat, 09 Aug 2008 18:35:00 +0000 http://caseelse.net/?p=36#comment-54 <p>Nice, Mark.</p> <p>The last time I actually wrote an Excel import script was for a manager who had already sent out several thousand questionnaires, and was starting to get them back. He couldn't figure out how to turn them into usable data.</p> <p>I wrote him a script (VBS, monitored a folder, and ripped any XLS that was dropped into it, cleaned it up and shoved it into Access.) Told him how to set it up, and how he'd have to fix the spreadsheets if the script spit them back at him, and that if he detailed anywhere it was consistently breaking, we could update the script.</p> <p>After that, he started asking for my opinion earlier in the decision tree.</p> Nice, Mark.

The last time I actually wrote an Excel import script was for a manager who had already sent out several thousand questionnaires, and was starting to get them back. He couldn’t figure out how to turn them into usable data.

I wrote him a script (VBS, monitored a folder, and ripped any XLS that was dropped into it, cleaned it up and shoved it into Access.) Told him how to set it up, and how he’d have to fix the spreadsheets if the script spit them back at him, and that if he detailed anywhere it was consistently breaking, we could update the script.

After that, he started asking for my opinion earlier in the decision tree.

]]> By: Neil http://caseelse.net/2008/08/06/why-not-to-use-excel-for-data-gathering/comment-page-1/#comment-52 Neil Sat, 09 Aug 2008 18:25:15 +0000 http://caseelse.net/?p=36#comment-52 <p>Spune, if you have any database people where you work, get their support. Update queries are easier (ie, less money spent on labor) than updating spreadsheets.</p> Spune, if you have any database people where you work, get their support. Update queries are easier (ie, less money spent on labor) than updating spreadsheets.

]]> By: Neil http://caseelse.net/2008/08/06/why-not-to-use-excel-for-data-gathering/comment-page-1/#comment-50 Neil Sat, 09 Aug 2008 18:22:22 +0000 http://caseelse.net/?p=36#comment-50 <p>Debra, I'm certainly not knocking Excel as a product. In fact, it's pretty handy as an output device (charts go over well with management) and it's easily automated.</p> <p>The article was in response to a couple of projects that consisted of sending out a <strong>lot</strong> (literally tens of thousands) of spreadsheets, which would have a fair number of free-text fields) and then importing the data on an ongoing basis.</p> <p>In this project, just the version overlap alone (ie, you come up with a glitch in the spreadsheet, you discover it will import better with a different value (or management changes something...); you still have thousands of spreadsheets floating around. There's no updgrade potential.) suits it for a centralized input device.</p> <p>For smaller, simpler (and certainly quicker) jobs, Excel is low-overhead, and low learning-curve.</p> Debra, I’m certainly not knocking Excel as a product. In fact, it’s pretty handy as an output device (charts go over well with management) and it’s easily automated.

The article was in response to a couple of projects that consisted of sending out a lot (literally tens of thousands) of spreadsheets, which would have a fair number of free-text fields) and then importing the data on an ongoing basis.

In this project, just the version overlap alone (ie, you come up with a glitch in the spreadsheet, you discover it will import better with a different value (or management changes something…); you still have thousands of spreadsheets floating around. There’s no updgrade potential.) suits it for a centralized input device.

For smaller, simpler (and certainly quicker) jobs, Excel is low-overhead, and low learning-curve.

]]> By: Neil http://caseelse.net/2008/08/06/why-not-to-use-excel-for-data-gathering/comment-page-1/#comment-49 Neil Sat, 09 Aug 2008 18:14:28 +0000 http://caseelse.net/?p=36#comment-49 <p>PC, I have nothing against Python...I haven't used it simply because it hasn't shown up as the right tool for the jobs I've done. (I will say, though, that flaming Perl isn't much of a reccomendation, in and of itself. How is Python in regards to parsing English text? I know it rocks at number-crunching. And how is Python for compiling into an EXE? That was one of the specific points.)</p> <p>BTW, I'm also not a Perl developer. The email was about *not* doing this sort of project in Excel, not about any given language.</p> PC, I have nothing against Python…I haven’t used it simply because it hasn’t shown up as the right tool for the jobs I’ve done. (I will say, though, that flaming Perl isn’t much of a reccomendation, in and of itself. How is Python in regards to parsing English text? I know it rocks at number-crunching. And how is Python for compiling into an EXE? That was one of the specific points.)

BTW, I’m also not a Perl developer. The email was about *not* doing this sort of project in Excel, not about any given language.

]]> By: Neil http://caseelse.net/2008/08/06/why-not-to-use-excel-for-data-gathering/comment-page-1/#comment-47 Neil Sat, 09 Aug 2008 18:09:09 +0000 http://caseelse.net/?p=36#comment-47 <p>Interesting thought, Ibrahim. And I'll have to look into it.</p> <p>It won't happen in the office, as it's a secure site. If infomration got posted on Google's servers, our security depeartment would have a well-earned cow <g></p> Interesting thought, Ibrahim. And I’ll have to look into it.

It won’t happen in the office, as it’s a secure site. If infomration got posted on Google’s servers, our security depeartment would have a well-earned cow <g>

]]> By: Kevin Merritt http://caseelse.net/2008/08/06/why-not-to-use-excel-for-data-gathering/comment-page-1/#comment-45 Kevin Merritt Sat, 09 Aug 2008 16:37:01 +0000 http://caseelse.net/?p=36#comment-45 This is one of the primary use case scenarios that we are solving at blist. We have created an online database that allows mainstream users to work in an environment that looks and feels like Excel, but the columns are "typed" - meaning the creator of the table can control the type of content people enter. We have checkboxes, T/F booleans that can be customized (hire/no-hire; pass/fail; go/no-go), picklists, etc. In addition to its ease of use, there's a huge benefit downstream in that it really is a database. You can analyze, print, chart, export, etc. We support import and export from and to Excel. It's free to use. Try it out at www.blist.com.   This is one of the primary use case scenarios that we are solving at blist. We have created an online database that allows mainstream users to work in an environment that looks and feels like Excel, but the columns are “typed” – meaning the creator of the table can control the type of content people enter. We have checkboxes, T/F booleans that can be customized (hire/no-hire; pass/fail; go/no-go), picklists, etc.
In addition to its ease of use, there’s a huge benefit downstream in that it really is a database. You can analyze, print, chart, export, etc.
We support import and export from and to Excel.
It’s free to use. Try it out at http://www.blist.com.
 

]]> By: Mark http://caseelse.net/2008/08/06/why-not-to-use-excel-for-data-gathering/comment-page-1/#comment-43 Mark Sat, 09 Aug 2008 15:30:49 +0000 http://caseelse.net/?p=36#comment-43 It's a knock-off for a scripter, too. And then another knock-off. And another. And another. Hey, you're right, pretty soon that is a career. I once worked with someone who was in charge of managing all the data for a team, who seemed to have the job only because she was incapable of doing anything else. She did everything with Excel, and there problems everywhere. What a mess. When I needed some data collected and processed, I had to go through her, according to the rules (she inserted herself in the pipeline between data and build). So I built a web app, as you suggest, for my portion of the data collection, and had her use that. She never got to touch the data, only the web UI. Worked like a charm.   It’s a knock-off for a scripter, too. And then another knock-off. And another. And another. Hey, you’re right, pretty soon that is a career.
I once worked with someone who was in charge of managing all the data for a team, who seemed to have the job only because she was incapable of doing anything else. She did everything with Excel, and there problems everywhere. What a mess. When I needed some data collected and processed, I had to go through her, according to the rules (she inserted herself in the pipeline between data and build). So I built a web app, as you suggest, for my portion of the data collection, and had her use that. She never got to touch the data, only the web UI. Worked like a charm.
 

]]>