Contrarian view: having a division between "general scientists" and "programmer scientists" is a good thing. I really don't miss the bad old days of scientists doing write-only code from a cobbled-together mess based on Numerical Recipes, leaving behind spaghetti C99 or F77 for the next post-doc, commenting out routines for version control; giving us papers with numerical analysis that's not even reproducible by the same group 12 months down the line.
In fact I think it's a much better thing to have a sort of division where a general scientist can put together a simulation/analysis pipeline in Python or R, but if they want to implement new algorithms they'll be sufficiently out of their league that they'll need help from someone who has actually spent time learning how to code properly and efficiently, to do testing and version control etc.
In fact it's very similar to how experimental science often works: you have the general scientist who just knows how to do basic measurements with some instrument, and you have the instrument scientist who has to help if the general scientist wants to use some novel technique, who keeps the equipment tidy and working and logs everything in the big lab journal.
I can understand how one can reasonably hypothesize that. In some sense you can see Julia as one big social experiment in what happens when you do everything possible to blur the line between users and developers of a scientific (and, of course, also general) programming language by solving the two language problem and making the language for users and developers one and the same.
... and it's been wildly successful. Some of the most prolific and talented contributors to Julia and its ecosystem are scientists by day and brilliant programmers of all kinds—crazy meta programmers, compiler hackers, generational GC writers, etc.—by night (as they say). During the keynote last night before tagging 1.0, we went through the history of major features in Julia's development history, it became a running theme that so many of these contributions have come from people who are physicists, chemists, geologists, biologists, etc. So I'd say that Julia is strong evidence that breaking down the separation between scientific users and developers of scientific libraries is a good idea.
I was watching live stream last night from the East coast and very impressed by the achievement by you scientists and grad students since 2012. https://arxiv.org/abs/1209.5145
As a former physicist, I have been through the Numerical Recipes/F77 stage myself two decades ago. In the age of Big Data and Machine Learning, there are many ways to harvest the creativities in people who are not trained as professional engineers. Our software development mindset should be extended beyond the industrial "task oriented" products and platforms.
In the spirit of Turing machine and LISP where code is also data, we can even treat scientific computing ecosystems like Python/R/Julia etc. as models to capture creative scientific minds. It is AI at a high social level. The future is beyond our imagination.
> I was watching live stream last night from the East coast and very impressed by the achievement by you scientists and grad students since 2012. https://arxiv.org/abs/1209.5145
Thank you, that's very kind of you! To be fair, Jeff, Viral, Alan and I had many collective decades of industry experience interleaved with an equal amount of academia by the time we wrote that paper. And of course the academics are rather suspicious about just how academic we really are.
That’s a great viewpoint, imho! Scientists in most fields have to write code nowadays, better to make it easier for them to hack together novel and useful tools.
In fact I think it's a much better thing to have a sort of division where a general scientist can put together a simulation/analysis pipeline in Python or R, but if they want to implement new algorithms they'll be sufficiently out of their league that they'll need help from someone who has actually spent time learning how to code properly and efficiently, to do testing and version control etc.
In fact it's very similar to how experimental science often works: you have the general scientist who just knows how to do basic measurements with some instrument, and you have the instrument scientist who has to help if the general scientist wants to use some novel technique, who keeps the equipment tidy and working and logs everything in the big lab journal.