Script Panel¶

In the ETA recipe, Script Panel provides the user interface for each experiment.

The common usage of Script Panel is to start an analysis on an existing time-tag file or a realtime time-tag sources, display results, and provide interactive controls of the acquisition device (time-tagger) or the analysis.

ETA provides Python API to allow customization of the Script Panel. Here we list all the API in the latest version of ETA. Please refer to the pre-made recipes for examples.

Note

When running the Script Panel, the entire recipe will be sent to ETA Backend. ETA will first check the Virtual Instruments and then start executing the code for the current Script Panel.

The code is executed in an isolated environment with an eta object and other user-defined parameters (only those in the same group of Script Panel will be visible) as global variables.

You can also import third-party Python libraries like numpy or matplotlib, etc.

Please note that ETA Backend can only run one Script Panel at the same time. If you have multiple ETA GUI connected to ETA Backend, the running requests will be put in a queue.

Prepare Timetag Clips¶

You need to prepare Clips of time-tags before running the analysis. Clips contian the actual timetag data, together with the metadata of the timetag data, like measurement resolutions, which are crucial to the correctness of an analysis. Clips can be loaded from a section of timetag file, or constructed from an exsiting buffer given by the timetagger library.

Due to the nature of the timetag data, we recommend using the Python generator function that yields Clips each time when it is called. ETA will automatically feed the new Clips from the generator when a current Clip reaches its end. ETA provides a set of APIs to help you build generators that automatically cut the file into a series of Clips. These APIs will read the header of the time-tag file and then set the Clip objects up, which can be later used in the analysis.

For users who are not familiar with generators, we use Python generator to implement the eta.clips() generator function, as a warpper of eta.clip_file(). And the eta.split_file() generator function will further warp the eta.clips.

You can also implement your own generator using Clip class or a lower-level API eta.clip_file. One example is that you can poll a timetagger library to see if there is new records in memory ready for analysis, when performing ETA streaming analysis.

eta.clip_file(filename, modify_clip=None, read_events=0, seek_event=-1, format=-1, wait_timeout=0)¶

eta.clip_file takes the previous Clip and number of events to be read as input, and generates a new Clip by sliding the cut window in the time tag file.

Note

The low-level API, eta.clip_file(), is the only way to actually load the timetag from file into memory and return only one Clip object for later use. You can think of eta.clip_file() as the timetag-specific way of doing read() in Python.

However, it is not recommended to use this low-level API directly, as it is complecated to manage multiple Clips during analysis. In order to keep the analysis running on a series of Clips, you will also need to put eta.clip_file() and eta.run together inside a for loop, with a task variable managed by yourself.

filename

The path to the time tag file. Please note that if you run ETA Backend on a remote computer, you need to specify the path to file on that computer.
modify_clip

If this parameter is None, ETA will read the file header and try to construct a new Clip object. If provided, ETA will modifed this previous Clip instead of creating a new Clip object from scratch by reading the header again, for better performance.

Note

When used together with seek_event=-1, ETA will slide the a window with length read_events in the time-tag file, starting from the ending position of the prevoius Clip. By iteratively feeding the returned Clip as modify_clip, one can fetch newly generated events from the file. It is useful for implementing real-time analysis, and there is a higher-level API, eta.clips, which does this automatically for you.

After calling this function, the modify_clip will be updated with timetag events from a new window in the timetag file. If you would like to keep the old Clip, please make a deep copy of the Clip object before calling this function.

If you would like to read a header-less file of a supported format, you could mannualy construct an empy Clip object with required format information, and feed that as modify_clip. Refer to Advanced Usage for more ideas.
read_events

The number of desired events to be loaded into the returned Clip. Setting it to 0 will make ETA read the entire file.

If the number exceeds the size of the time-tag file, and wait_timeout is configured, ETA will wait until the file grows to that size. This is particularly useful in a real-time analysis where the time-tag file is continouslys growing.

If file failed to grow to the desired size, a shortened Clip to the current ending of the file will be loaded. If no records can be loaded at all, a False will be returned indicating a failure, and the modify_clip will not be modified.

Note

You can also set a negative value, then the number of records in this Clip will be calculated as the number of records between the ending of the last Clip to the current end of file minus the absolute value of this negative number.

The time tag file that serves as the FIFO when you perform a real-time analysis might have pre-allocated-but-not-yet-written areas, and the negative value here can help you get rid of that.
seek_event

Setting the starting event number for reading. Setting to it 0 will force ETA to read from the first event after the file header.
wait_timeout

Value in seconds specifies the maximum waiting time. ETA will wait until the file grows to desired size. If file failed to grow to the desired size, a shortened Clip to the current ending of the file will be loaded.
format

Format specifies the time-tag file format that you want to use in the analysis. The default is set to the auto detection of PicoQuant devices. You can also use the constant eta.FORMAT_SI_16bytes for Swabian Instrument binary format, eta.FORMAT_QT_BINARY for qutools quTAG 10-byte Binary format, eta.FORMAT_QT_COMPRESSED for compressed qutools quTAG binary format, or eta.FORMAT_BH_spc_4bytes for Becker & Hickl SPC-134/144/154/830 format, or eta.FORMAT_ET_A033 for Eventech ET A033 format.

Note

The format of time-tag you use might influence the analysis results in unexpected ways, through the nature of timing system it uses.

If the timetag file is recorded with absolute timing (default for most of the time taggers), then every cut should keep the same absolute timing.

If the timetag file is recorded with relative timing (like in HHT3 mode), then the absolute timing for each cut will take the first event in this cut as the reference of zero. You should be extremely careful when using seek_event to seek to arbitrary position, as the file format supports only continuous sequential read.

eta.clips(filename, modify_clip=None, read_events=1024102410, seek_event=-1, format=-1, wait_timeout=0, reuse_clips=True, keep_indexes=None)¶

eta.clips makes a generator that yields Clips with a specified amount of new record read from the file. It is wrapper on top of eta.clip_file(). Instead of returning only one Clip object, it will return a generator that yields a Clip every time it called. It inherts most of the parameters from eta.clip_file(), and also adds some new parameters.

read_events

This amount of events will be read for each Clip that this generator yields.
seek_event

Setting the starting event number for reading the first clip. This parameter will be ignored starting from the second clip, as the second second clip would slide a window of read_events. If you want to skip some windows, use keep_indexes instead.
reuse_clips

If set to False, the previous Clip will not be modifed, and a new Clip will be created everytime it is called.

Note

This is useful when you want to load all the Clips at once. For example, in a correlational analysis, we can set this parameter to False, and then use list(ret) to load the file into some equal-size Clips in a list, with which you could run parallel analysis to get speed boosts.

Please be careful when setting this to False, as it may cause memory leaking if the references are not handeled properly.
keep_indexes

A list of indexes of the sliding windows for Clips that will be actually yielded. Other Clips will be skipped. Indexes start from 0 for the first window of [0,read_events], and index 1 means [read_events,read_events*2].

eta.clips_list(filename, read_events=10241024, format=-1, threads=os.cpu_count()2)¶

eta.clips_list makes a list of generators using eta.clips for parallel analysis. It sets the keep_indexes or seek_event automatically for each of the generators, so that the file is splited into roughly equal sizes for each generators.

Unlike eta.split_file, which makes one single generator that splits the file into a certain number of equal size Clips and yield them one by one, eta.clips_list would return a list of generators, and each of them can eta.clips the file into a configurable read_events size.

threads

How many threads you want to use. It will decide how many sections you want to split the file into, and how many clips generators are returned.

eta.split_file(filename, modify_clip=None, cuts=1, format=-1, wait_timeout=0, reuse_clips=True, keep_indexes=None)¶

DEPRECATED eta.split_file is simple wrapper on top of eta.clips(), that makes a generator that splits the file into a desired amount of equal size Clips. It inherts most of the parameters from eta.clips().

cuts

The number of Clips that you want to generate. Default value is set to 1, thus the full time-tag will be returned in one cut descriptor.
keep_indexes
A list of indexes of the sliding windows for Clips that will be actually yielded. Other Clips will be skipped. Indexes start from 0 for the first window of [0,read_events], and index 1 means [read_events,read_events*2].

Examples:
#stop evaluation of timetag stream after 2% cutfile = eta.split_file(file,100,keep_indexes=[1,2]) result = eta.run(cutfile)

Executing Analysis¶

eta.run(sources, resume_task=None, group=”main”, return_task=False, return_results=True, background=None, max_autofeed=0, stop_with_source=True)¶

eta.run() starts an analysis, where you actually feed all sources into RFILES in Virtual Instruments and obtain results.

You can use Python generators functions, that yields Clip objects, as a source. ETA will do auto-feeding , fetching one new Clip from the generator each time, so the generator functions will be called many times.

In a single invoke of eta.run(), only a single task will be used for all Clips generated by the generator, until the generator reaches its end or max_autofeed is reached. By default eta.run will use a new task for the analysis, unless resume_task is specified.

The analysis will block the execution of Python script until the results are returned in a Python dictionary, with the ``HISTOGRAM`` names as the keys. If you want to schedule many analysis and run them in parallel, you can set return_results=Fasle. Clip objects indicating the current reading positions for all generators in the sources, will also be returned in the results dictionary with the RFILE names as keys.

sources

A dict of Python generators functions that yields Clips. The keys should match with the name of coresponding RFILEs in the virtual instrument.

If only one generator is provided instead of a dict, it will be distributed to all RFILEs, which might cause unexpected behaviors.
max_autofeed

It limits the number of Clips that eta.run would fetch from the generator.

Set this value to 1 if you want to get each result for every single Clip from the generator, rather than get final result after the full generator is consumed.
background

Run the analysis in the background. Set it to True will start a new therad in the thread pool. By default background=not return_results

In this case, you must turn on return_task so that the task descriptor will be returned immediately, and the analysis will continue running in the background. You can start many threads in the background and gather a list of task descriptors, with which you can aggregate the results from these threads later.
return_results

Specifies if a dictionary of results should be returned.

This is the switch for multi-threading analysis. No new thread will be created and the analysis will be performed in MainThread if this option is set to True.

Note

The parameter for enabling multi-thread mode is removed since version 0.6.6, when we switch to the Map-Reduce style of multi-threading. The new way of doing multi-threading is easier and more flexibile. eta.run works like Map, and eta.aggregrate works like Reduce.

You can schedule your analysis from Script Panel in any way you want. As long as you keep the task descriptor, you will be able to retrieve the result in the end.
return_task

Specifies if the modified task descriptor should returned.

You must set it to True if return_results is set to False. If both of them are set to Ture, you can get both of them with result, task = eta.run(..., return_task=True, return_results=True), and later you can resume an analysis with the task descriptor using resume_task.

Note

The context parameter is renamed to task descriptor to reduce confusion since version 0.6.6.

Task descriptor works like a complete memory snapshot of a current running or finished analysis. Everything except for the sources (Clips) is preserved. If you want to reprod can resume the analysis later without worrying about underlying details of the anlaysis.
group

The group name of instruments that you want to run analysis on. This parameter is provided so that eta.run can automatically call eta.create_task using the provided group name when resume_task is not provided.
resume_task

Specifies an existing task descriptor to resume the analysis.

A new task descriptor can be created with eta.create_task. You can also iteratively call eta.run using the returned task from a previous eta.run call. In second case, the analysis will be resumed from the point where it ended, with all contexts set correctly, and then feeded with the new Clip. This is particularly useful when you want to perform real-time or streamming analysis.

Note

After the analysis is resumed, the old task descriptor becomes invalid, however, a new task descriptor can be returned by setting return_task=True.

The way how the files is cut into Clips, or the order in which eta.run is invoked, will never affect analysis result, as long as you always resume with the last task descriptor (or None for the first iteration) during the entire analysis.

In multi-threading analysis, however, there will usually be the same amount of “last” task descriptors missing during the fisrt iteration, as the number of threads you use. You will also end up with that amount of task descriptor in the end. For some analysis, like correlation which yields histograms, you can use eta.aggregrate later to merge the analysis results from those tasks into one. But it won’t change the fact that they are essentialy many different independent analysis.
stop_with_source

Stop the analysis when any of the sources reaches its end. Set it to False if you want to run simulation without any source.

eta.create_task(group, resume_task=None):¶

eta.create_task will create a new analysis task using the group of instruments. The returned task can be used in eta.run(resume_task=task) The instruments within the same group are visible to each other in this task.

group

The group name of instruments that you want to run analysis on.
resume_task

Same as in eta.run().

eta.aggregrate(list_of_tasks, sum_results=True, include_timing=False):¶

eta.aggregrate will gather data form previous multi-threading anlaysis tasks started with return_results=False and put them together as the final results. If all previously anlaysis tasks haven’t finished, ETA will block until all of them are finished.

list_of_tasks

A list of previously created task descriptors, from which you want to retrieve results.

Note

You can run multi-threading analysis on different groups for completely different analysis at the same time. However, you can only aggregrate the results using form task descriptors created by eta.run on the same group.
sum_results

Specifies if the results will be summed up.

Note

This is useful for correlational analysis if you want to merges histograms from many individual analysis tasks. Keep in mind that you will need to make sure that it is physically meaningful to perform adding. (Is the histogram in the same base unit? Can you add histograms from experiments done today and yesterday? Will the result be different from running with with only one task, but many Clips instead.)

Users can also set this value to False and get a list of dict returned instead. Then they can use their own data aggregation methods, like concatenating to generate large images.
include_timing

Specifies if the timing information eta_total_time, eta_compute_time, max_eta_total_time, max_eta_compute_time should be appended into the results.

Examples:

clipgens=eta.clips_list(file)
# assign different index_range to different clip generators, so that they read different parts of the original file
tasks = []
for cutfile in clipgens:
    tasks.append( eta.run({'UniBuf1':cutfile}, group='compile',return_task=True,return_results=False))
    # start a thread in the background for each clip generator
    # and keep the reference to the task descriptor
results = eta.aggregrate(tasks,include_timing=True)

Interacting with ETA GUI¶

eta.display(app)¶

You can send results to ETA GUI using this function. The value of app can be either a Dash or Bokeh graph currently.

Note

Use app = dash.Dash() to create a Dash graph.

logging.getLogger(‘etabackend.frontend’)¶

Returns the logger, with which you can display inforamtion on the GUI.

logger.info(msgstring)¶

This is the ETA alternative for print() in Python. This is useful when you use want to display some message on the ETA GUI.

msgstring

Message string to be shown on the ETA GUI.

logger.setLevel(loglevel)¶

This modifies the logging level of a specific logger.

loglevel

A loglevel from logging. Can be logging.WARNING

Examples:

logger = logging.getLogger('etabackend.frontend')
logger.info('No further logoutput for the realtime recipe.')
logger.setLevel(logging.WARNING)
plt.show()
logger.setLevel(logging.INFO)

eta.send(text,endpoint=”log”):¶

This is useful when you want to talk to another program other than ETA GUI via WebSocket (see Advanced Usages). You can stream the results back using this function.

text

String of information to be sent to the client.
endpoint

Can either be log or err, for indicating the type of message.

Modify recipies programatically¶

You can also modify recipes programmatically. The recipe uploaded will be availble under eta.recipe.

As an example, you can upload the template recipe from your LabVIEW program to ETA Backend via WebSocket (see Advanced Usages), and then change the parameters (like bin size for histograms) to get different results.

eta.load_recipe(jsonobj=None)¶

Converting the jsonobj to a Recipe object and save it into eta.recipe. Then refresh compling cache if eta.recipe is modified.

jsonobj

A JSON object parsed from .eta file. If not provided, the current eta.recipe will not be modifed.

eta.recipe.get_parameter(name)¶

Get the value of a parameter in the recipe, given the name of the parameter. If there are multiple parameters with the same name, only the first one will be returned.

name

Name of the parameter, as shown in the ETA GUI.

eta.recipe.set_parameter(name, value=None, group=None)¶

Set the value (and group) of parameters in the recipe, given the name of the parameters.

name

Name of the parameter, as shown in the ETA GUI.
value

The new value of this parameter. If None is given, the value will not be modified.
group

The new group of this parameter. If None is given, the group will not be modified.

Note

The updated parameters will be applied to the next Run. Call eta.load_recipe() after finishing updating parameters and before eta.run if you want to apply it immediately.

eta.recipe.del_parameter(name)¶

Delete a parameter in the recipe, given the name of the parameter. If there are multiple parameters with the same name, only the first one will be deleted.

name

Name of the parameter, as shown in the ETA GUI.

Using Third-party Libraries¶

The following libraries are required to be installed with ETA. Feel free to use them in your recipes.

numpy
scipy
lmfit
matplotlib
dash
dash-renderer
dash-html-components
dash-core-components
plotly
bokeh

Using other third-party libraries (including Python libraries or dynamic linked libraries) might lead to not fully portable recipes. Please distribute the libraries with the recipe, so that the users can download and install them. ETA also recommends distributing the libraries on ETA-DLC (ETA downloadable contents).

Script Panel¶

Prepare Timetag Clips¶

eta.clip_file(filename, modify_clip=None, read_events=0, seek_event=-1, format=-1, wait_timeout=0)¶

eta.clips(filename, modify_clip=None, read_events=1024*1024*10, seek_event=-1, format=-1, wait_timeout=0, reuse_clips=True, keep_indexes=None)¶

eta.clips_list(filename, read_events=1024*1024, format=-1, threads=os.cpu_count()*2)¶

eta.split_file(filename, modify_clip=None, cuts=1, format=-1, wait_timeout=0, reuse_clips=True, keep_indexes=None)¶

Executing Analysis¶

eta.run(sources, resume_task=None, group=”main”, return_task=False, return_results=True, background=None, max_autofeed=0, stop_with_source=True)¶

eta.create_task(group, resume_task=None):¶

eta.aggregrate(list_of_tasks, sum_results=True, include_timing=False):¶

Interacting with ETA GUI¶

eta.display(app)¶

logging.getLogger(‘etabackend.frontend’)¶

logger.info(msgstring)¶

logger.setLevel(loglevel)¶

eta.send(text,endpoint=”log”):¶

Modify recipies programatically¶

eta.load_recipe(jsonobj=None)¶

eta.recipe.get_parameter(name)¶

eta.recipe.set_parameter(name, value=None, group=None)¶

eta.recipe.del_parameter(name)¶

Using Third-party Libraries¶

eta.clips(filename, modify_clip=None, read_events=1024102410, seek_event=-1, format=-1, wait_timeout=0, reuse_clips=True, keep_indexes=None)¶

eta.clips_list(filename, read_events=10241024, format=-1, threads=os.cpu_count()2)¶