file io - Python gzip.open .tell() has a linear increasing factor making it slow -

-

using python 3.3.5, have code looks like:

with gzip.open(fname, mode='rb') fh:     fh.seek(savedpos)     line in fh:         # work done         savedpos = fh.tell()

the work being done on each row quite taxing on system, wasn't hoping great numbers. threw in debug counter , got following result:

48 rows/sec 28 rows/sec 19 rows/sec 15 rows/sec 13 rows/sec 13 rows/sec 9 rows/sec 10 rows/sec 9 rows/sec 9 rows/sec 8 rows/sec 8 rows/sec 8 rows/sec 8 rows/sec 7 rows/sec 7 rows/sec 7 rows/sec 7 rows/sec 5 rows/sec ...

which tells me off, put fh.tell() in debug-counter/timer function, making fh.tell() executed once second , got stable 65 rows/sec.

am off shelf or shouldn't fh.tell() extremely quick? or side-affect of gzip alone?

i used store file-position manually bugged out due different file-endings, encoding issues etc figured fh.tell() more accurate.

are there alternatives or can speed fh.tell() how?

my experience zlib (albeit using c rather python, suspect issue same) seeking slow. zlib doesn't keep track of in file is, if seek has uncompress beginning in order count how many uncompressed bytes forward should seek to.

in other words, reading or writing sequentially fine. if have seek, you're in world of hurt.


Comments

Post a Comment

Popular posts from this blog

java - Oracle EBS .ClassNotFoundException: oracle.apps.fnd.formsClient.FormsLauncher.class ERROR -

-
i need in configuring java on machine because, when try loading forms, loader goes in circles , gives error: java.lang.classnotfoundexception: oracle.apps.fnd.formsclient.formslauncher.class @ sun.plugin2.applet.applet2classloader.findclass(unknown source) @ sun.plugin2.applet.plugin2classloader.loadclass0(unknown source) @ sun.plugin2.applet.plugin2classloader.loadclass(unknown source) @ sun.plugin2.applet.plugin2classloader.loadclass(unknown source) @ java.lang.classloader.loadclass(unknown source) @ sun.plugin2.applet.plugin2classloader.loadcode(unknown source) @ sun.plugin2.applet.plugin2manager.createapplet(unknown source) @ sun.plugin2.applet.plugin2manager$appletexecutionrunnable.run (unknown source) @ java.lang.thread.run(unknown source) i have tried re-installing firefox, older versions of java runtime environments, disabling proxy. what missing? application working on other machines. i spent around 100 hours finding s
Read more

c# - how to use buttonedit in devexpress gridcontrol -

-
Image
i trying create gridcontrol buttonedit in 1 of columns. when user clicks button edit, popups form select product. when selection done on popup, returns datarow selection main grid below. but when column loses focus, value wrote column disappear. here code creates gridcontrol's data , buttonedit's click event. private void frmsiparisnew_load(object sender, eventargs e) { dt = new datatable(); dt.columns.add("malzeme_kodu",typeof(string)); dt.columns.add("malzeme_aciklama", typeof(string)); dt.columns.add("adet", typeof(decimal)); dt.columns.add("birim", typeof(string)); dt.columns.add("fiyat", typeof(decimal)); dt.columns.add("kur", typeof(string)); dt.columns.add("tutar", typeof(decimal)); datarow dr = dt.newrow(); dt.rows.add(dr); gc.datasource = dt; } private void repositoryitembuttoneditmal
Read more

nvd3.js - angularjs-nvd3-directives setting color in legend as well as in chart elements -

-
good morning i using angularjs-nvd3-directives, great, having difficulties when setting color of multi bar chart ( http://cmaurer.github.io/angularjs-nvd3-directives/multi.bar.chart.html ). the show legend works well: <div ng-controller="examplectrl"> <nvd3-multi-bar-chart data="exampledata" id="showlegendexample" width="550" height="300" showxaxis="true" showyaxis="true" xaxistickformat="xaxistickformatfunction()" showlegend="true"> <svg></svg> </nvd3-multi-bar-chart> </div> but when add option in conjunction color="colorfunction()" where, var colorarray = ['#ff0000', '#0000ff', '#ffff00', '#00ffff']; $scope.colorfunction = function() { return function(d, i) { return colorarray[i]; }; } the color of cha
Read more