Scientists and engineers: python advantages

Discussion in 'Programmer's Corner' started by someonesdad, Nov 13, 2010.

  1. someonesdad

    Thread Starter Senior Member

    Jul 7, 2009
    1,585
    141
    A week or two ago a discussion in another thread made me think about writing a document as to why the programming language python and its add-ons are good tools for a technical person to have in his toolkit. I'm not suggesting it should take the place of other languages (indeed, an experienced technical person will know many languages), but I've found it an excellent general-purpose programming tool and it is usually what I turn to first.

    I wrote the attached document to introduce some of the features of the language and discuss some of the add-ons that provide extra power. You can also get the zip file here (see the bottom of the page).

    Edit 17 Nov 2010: I updated the document to add another example and discuss the problem of choosing between python 2.x and 3.x in more detail. The latest version will always be here: http://code.google.com/p/hobbyutil/downloads/list. I recommend getting the file from this link rather than using the file attached to this post.
     
    Last edited: Nov 17, 2010
  2. tom66

    Senior Member

    May 9, 2009
    2,613
    214
    Very nice!

    I like Python lots, and I think it should really be a standard language seen more often.
     
  3. rapidcoder

    Member

    Jan 16, 2011
    37
    3
    What has Python that Scala doesn't have? :)
    Scala > Python in almost every aspect, IMHO.
     
  4. someonesdad

    Thread Starter Senior Member

    Jul 7, 2009
    1,585
    141
    Very possibly. Back up your statement with a comparative document; otherwise, you have zero street cred.
     
  5. rapidcoder

    Member

    Jan 16, 2011
    37
    3
    1. Scala is statically typed. Python has no static types. Which means if you make a type error, it will fail at runtime. If you make an error in a rarely taken branch, you may not notice the error when testing. In Scala such errors are detected on the program startup.
    2. Scala is about 20-100x faster than Python to execute. Python is interpreted, Scala is compiled to machine code.
    3. Scala has pattern matching, Python has nothing comparable.
    4. Scala gives you more freedom in how you lay out your code, there is no meaningful whitespace.
    5. Scala IDEs have code completion and refactoring support. This is not possible for Python.
    6. Scala code is usually just as short and concise as Python (or sometimes shorter), albeit faster to type in because of auto code completion.
    7. Scala has more mature libraries (any Java libraries work with it).

    Enough?

    Ok, Python has one advantage over Scala:
    1. It is better for scripting short tasks. Scala scripts just take too much time to start.
     
  6. tom66

    Senior Member

    May 9, 2009
    2,613
    214
    Dynamic typing brings advantages, and disadvantages. For example, you can create methods which can handle any type of members. Say an addition function; it can handle ints, floats, and any other type of item which can be added to another (as long as they are compatible - for example, adding an int and a list doesn't make sense, and will rightly raise an exception.)

    Heard of Psyco? Python code can run in a JIT, which is almost as fast as native C! For most applications speed isn't crucial. If you need speed, you shouldn't be writing in either.

    See module "re" for regex, "glob" and "fnmatch" for basic wildcard (Unix-style) matching.

    Python will let you write in any whitespace style you like, and frankly, you should be using whitespace anyway.

    List of Python IDEs.

    See above.

    JPython or Jython lets you run Java libraries from Python and the Python libraries are pretty good in my opinion anyway.
     
  7. rapidcoder

    Member

    Jan 16, 2011
    37
    3
    Some statically typed languages can do this too. Scala can.

    It is not. See the benchmarks. Actually it is only slightly fastert than the interpreted Python. The problem is not interpretation - the problem is lack of static types which makes calling methods and accessing fields very costly. Jython is a variant of Python that runs on JVM, and it is JITed. However, it is still much, much slower than Java, Scala or Groovy - because they are statically typed.

    Blah. Pattern matching is not textual regular expressions. It is a much more powerful thing. Pattern matching is a general concept common to functional programming languages - it allows you to match objects against object patterns. Not just textual or sequential objects. Used in ML family of languages, Haskell and Scala (and probably some others).

    Ok point taken. However I still don't believe the tabs vs space problem won't bite you at some time.

    Nice there are lots of IDEs, but no really working code completion. Most on that list are beta quality.

    Agreed. Point taken. I forgot about Jython. However, as for JVM languages it doesn't stand a chance with Clojure and Scala. :)
     
  8. someonesdad

    Thread Starter Senior Member

    Jul 7, 2009
    1,585
    141
    <pulls python from its holster> Ya gotta ask yourself one thing: "Do I feel lucky?" Well, do ya punk?
     
  9. rapidcoder

    Member

    Jan 16, 2011
    37
    3
    Some statically typed languages can do this too. Scala can.

    It is not. See the benchmarks. Actually it is only slightly fastert than the interpreted Python. The problem is not interpretation - the problem is lack of static types which makes calling methods and accessing fields very costly. Jython is a variant of Python that runs on JVM, and it is JITed. However, it is still much, much slower than Java, Scala or Groovy - because they are statically typed.

    Blah. Pattern matching is not textual regular expressions. It is a much more powerful thing. Pattern matching is a general concept common to functional programming languages - it allows you to match objects against object patterns. Not just textual or sequential objects. Used in ML family of languages, Haskell and Scala (and probably some others).

    Ok point taken. However I still don't believe the tabs vs space problem won't bite you at some time.

    Nice there are lots of IDEs, but no really working code completion. Most on that list are beta quality.

    Agreed. Point taken. :)
     
  10. tom66

    Senior Member

    May 9, 2009
    2,613
    214
    Okay.. Point taken, but there are many other uses for dynamic/"duck" typing.

    It depends on exactly what you are doing. If you're doing algorithmic stuff, you can get very high performance. If you're doing lots of method calls, and mixing types, it will be slow.

    I must admit I have never heard of this before. I have no idea if there are Python libraries or native support for it.

    Stick with spaces and there will be no problems. In many IDEs, it's possible to get the editor to replace tabs with a number of spaces.

    IDLE has code completion, so does NetBeans, as does PyDev, so does WinGlde... in fact, most on that list do! However, I have never used code completion and never found a use for it.

    I have nothing to add as I cannot compare languages I have not used - I am only raising points about my personal favourite.
     
  11. rapidcoder

    Member

    Jan 16, 2011
    37
    3
    And what prohibits you from having both static typing and duck typing in the same language? Google Go and Scala are languages that show it is possible. Google Go has exact duck typing, as in dynamic languages, Scala has an enhanced, type safe version of duck typing, called structural typing. Both can do whatever is possible for dynamic languages.

    Doing complex algorithmic stuff (like creating a simulation engine) means usually doing lots of method calls, creating lots of classes and objects, etc. It is easy to go high-performance in a microbenchmark, it is not easy in a large application if method calls - a basic glue - are slow and not inlined.

    There is a big difference between having it on the list, and actually having it working dependably. I have friends who write lots of Python, and they also don't use code completion - because it almost never works, except in trivial cases, where it is not needed. Code completion is great, especially if you work with code that you haven't written. Saves lots of documentation lookups. Types are documentation, a kind of documentation that is always up to date, so very very valuable.

    Someone created a very good blog post on type systems:
    http://www.pphsg.org/cdsmith/types.html
     
  12. tom66

    Senior Member

    May 9, 2009
    2,613
    214
    This is interesting, I found this on Wikipedia (not sure if it is related):

    Personally, I do not understand structural typing, I need to take a closer look at it.

    I'm thinking more like sequential expressions - for example, in one project in Python there were about 20 lines of code which manipulated a matrix, and despite using classes, it did speed up considerably with Psyco (probably around 10x, but I didn't record it.) And I made a program which did a lot of looping - typically something that should be slow - and it was about 20x faster with Psyco.

    As I said - I do not use it and have never found a use for it in any language. So I can't offer much comparsion.
     
  13. tomson

    New Member

    Sep 6, 2010
    16
    0
    as for code completion - I'm using PyCharm (it's not free though) and it works just fine. comodo also looks nice but haven't tried it.

    I agree to most of the points apart of performance - you want performance, than stay away from anything related to Java...still.
     
  14. rapidcoder

    Member

    Jan 16, 2011
    37
    3
    Can you back up your opinion with any code or real-world example?

    I can see lots of evidence Java's performance is on par with C now, if you take enough large application and exclude stupid microbenchmarks of the type "Look ma, I worked on these 10 lines of C code for a week, it is a terribly unreadable mess now, but works 10% faster than Java".

    Some real-world examples:
    - Facebook decided to write their high performance database system in Java.
    - Amazon also uses pure Java solution for their core database.
    - Twitter created their message routing backend in Scala (JVM language), mainly due to performance and scalability considerations.
    - Jake2, a Quake2 port to Java is just as fast as the original
    - Java Hadoop won some high performance computing competitions, outperforming software written in C++
    - Tomcat is just as fast as the Apache webserver, Netty is very close to Nginx.
    - HSQLDB or H2 database engine compared to MySQL and PostgreSQL. The C/C++ databases are definitely slower than their Java counterparts.

    And one more, taken from my personal experience:

    We've recently performance tested our circuit simulation engine prototype running on JVM doing some RLC filter simulation and we found it 2-3 times faster than CircuitLogix, TopSpice and Tina (all written in C). It is still a little slower than LTSpice, however we are still in the alpha stage - the project is only a half year old. I think we are able to match LTSpice in performance soon - a 2x performance improvement is easily possible, but not our priority now. BTW: LTSpice doesn't guarantee global convergence for non-linear circuits, while ours does. Doing things well is more important than doing things fast. :p

    The problem with writing big and fast software in C or C++ is that you can either go big, or go fast, but not both, unless you have unlimited budget and time. So while it is easy to outperform Java in a small microbenchmark, it is much harder to do it for a real world big application. Simply, when you get big enough, you have little time left for optimisation - most of the time is spent creating features and fixing bugs. And at this Java is much better than C family languages, especially if we are talking about hard, complex scientific code.

    It is <snip> easier to implement good and complex algorithms in Java, Scala or Python than it is in C++. C++ programmers struggle often to get just ANY solution of their problem, because the language adds too much own complexity to it. When moved to a more expressive language, the problem seems simpler and now you can not only think of ANY solution, but also of a FAST solution and optimal algorithm. This is the case with our simulator and SPICE. Ours is not faster because we have implemented the same SPICE algorithms better in Java. We have just used different, better algorithms.
     
    Last edited by a moderator: Mar 22, 2011
  15. cj3

    New Member

    Mar 9, 2012
    11
    0
    The problem with Python is two-fold: 1. Its syntax (obfuscated C); and 2. Weak typing. A better thought out syntax is Ada; and strong typing should be forced, such as in Ada or TrueBASIC.
     
  16. RobertB

    New Member

    Jun 5, 2012
    2
    1
    Python was my first programming language.
     
  17. ingframin

    New Member

    Feb 2, 2012
    1
    0
    Python and Java are different languages with different purposes.
    In an electronic lab the biggest programming activity is scripting to automate measurements or analyzing data.
    Python for this is perfect because Java requires a good object oriented design in order not to have terrible code and it's not suited for scripting.
    Consider also that loading in Java the VISA driver or the Daqmx driver is really complicated...
    With Python this is easy and we can go way faster in developing our scripts than with Java.
    I love Java, but I use it for other purposes :)
    I think it's a shared opinion that the best is to know both languages (and possibly some other language) in order to make the best choice depending on the case.
    About speed problem... I must admit that usually the main speed problem is between the chair and the keyboard ;) (but mobbing my friends that are computer scientists I can solve a big part of this issues :p )
     
  18. nwala emmanuel

    New Member

    Oct 29, 2012
    1
    0
    i need somebody that can convert 5-10volts to 210-220 their is something i want to invent please i need a reply or add me on Facebook,www.facebook.com/nwala.emmanuel1
     
  19. takao21203

    Distinguished Member

    Apr 28, 2012
    3,577
    463
    That is what I was thinking when I examined it. Some kind of weird C. Same with PHP, and Ruby. Yes I know these are interpreted, dynamic, scripted languages.

    I found it difficult enough to keep up with these new C++ extensions, and at the same time, also to use embedded C.

    If you look at websites these days, they are so full of unneccessary things...for sure, hordes of script language programmers are employed, because it is so difficult to use these languages.

    I don't really understand what is so new or revolutionary with these script languages.
     
  20. KL7AJ

    AAC Fanatic!

    Nov 4, 2008
    2,040
    287
     
Loading...