bash script strategies

 I did a fair amount of bash scripting for a recent project to provide support for managing an ecosystem of docker containers, java applications, and javascript server, including some low-level utility scripts that are shared by higher-level scripts. 

In the process, I realized that it would be helpful to develop and follow some rules and patterns because otherwise it would be easy to have different solutions to the same problem in different places in code that is already difficult to read and understand. 

To that end, I decided to create a document on the project wiki to describe my current thinking and approach on various aspects of bash script development (portability, defining variables, command substitution, passing and receiving parameters between scripts, testing conditions, checking results of commands, console output, ...) so that I can refer back to it and extend it when I have similar work to do in the future.

disable App Store notifications on MacOS

I get an App Store notification on my MacBook literally every time I use it.  The sound makes my dogs go crazy, because it's the same noise as the camera on the front door bell.  Of course in true Apple fashion, there is no "App Store" notification setting on "System Preferences".  The only "solution" I can find is to allegedly disable notifications until some future date.  E.g. open Terminal and enter something like this:

defaults write com.apple.appstored LastUpdateNotification -date "2099-12-12 12:00:00 +0000"

mysql stored function with recursive query

I recently had a requirement for an advanced search/filter mechanism for an entity in a JSF application outside the scope of the relevant database entity's regular attributes.  It required a recursive query, as the database table represents a hierarchy of the items that it contains.  Our database backend is mysql/mariadb 8.

I decided to use a mysql stored function for the recursive query because I wanted to use the query from arbitrary SQL in contexts than the original requirement.  I have used recursive queries or stored functions in mysql before, so I just wanted to document my approach here for future reference.

I found the following links helpful:

• https://mariadb.com/kb/en/recursive-common-table-expressions-overview/
• https://www.geeksforgeeks.org/mysql-creating-stored-function/

The recursive part of the query looks like this:

WITH RECURSIVE ancestors as ( 

     SELECT parent_item_id FROM item_element WHERE parent_item_id in ( 

          SELECT parent_item_id FROM item_element WHERE id in ( 

                 SELECT first_item_element_id  

           FROM item_element_relationship ier  

           WHERE ier.relationship_type_id = 4 AND  

        second_item_element_id = item_self_element_id)) 

     UNION  

           SELECT ie.parent_item_id  

           FROM item_element ie, ancestors AS a  

           WHERE ie.contained_item_id1 = a.parent_item_id 

) SELECT count(i.name)  

INTO row_count  

FROM item i, ancestors  

WHERE i.id = ancestors.parent_item_id and i.name like name_filter_value;

The first part of the query (before the union) is the "anchor".  It is executed first and selects the elements that for the cable's direct endpoint machine design items.  The "recursive part" (after the union) is executed repeatedly until it returns no new data.

The function accepts parameters for the name filter value, and an optional specification of cable end for constraining the result to one end of the cable or the other.  Example queries look like this:

-- don't constrain cable end, matches any name against any device ancestor

select * from item where domain_id = 9 and cable_design_ancestor_filter(id, 'DLMA', '');

-- constrain matches by cable end (1 or 2)

select * 

from item 

where domain_id = 9 

and cable_design_ancestor_filter(id, 'Rack 01-01', '1') 

and cable_design_ancestor_filter(id, 'DLMA', '2');

Also worth documenting here is the mechanism for calling a stored function from JPA.  This is done by using "FUNCTION()" in the SQL statement, as is shown in these two named query examples:

@NamedQueries({

    @NamedQuery(name = "ItemDomainCableDesign.filterAncestorAny",

            query = "SELECT i FROM Item i WHERE i.domain.name = :domainName AND FUNCTION('cable_design_ancestor_filter', i.id, :nameFilterValue, '')"),

    @NamedQuery(name = "ItemDomainCableDesign.filterAncestorByEnd",

            query = "SELECT i FROM Item i WHERE i.domain.name = :domainName AND FUNCTION('cable_design_ancestor_filter', i.id, :end1Value, '1') AND FUNCTION('cable_design_ancestor_filter', i.id, :end2Value, '2')")

})

Here is the full code of the stored function. One thing I tried to avoid was that there are two separate recursive queries for the conditional cases where cable end either is or is not specified.  I was unable to find a solution using a stored procedure or by concatenating strings to form the query, but I'm guessing there must be a way...

DROP FUNCTION IF EXISTS cable_design_ancestor_filter// 

CREATE FUNCTION cable_design_ancestor_filter 

(cable_item_id INT, 

name_filter_value VARCHAR(64), 

cable_end VARCHAR(64)) 

RETURNS BOOLEAN 

BEGIN  

DECLARE row_count INT; 

DECLARE item_self_element_id INT; 

-- check that cable_item_id is specified

IF ISNULL(cable_item_id)

THEN

SIGNAL SQLSTATE '45000'

SET MESSAGE_TEXT = 'cable item id must be specified';

END IF;

-- add % to both ends of filter value 

IF ISNULL(name_filter_value) OR name_filter_value = ''

THEN

RETURN true;

END IF;

SET name_filter_value = CONCAT('%', name_filter_value); 

SET name_filter_value = CONCAT(name_filter_value, '%');

-- get self element id for cable design item 

SELECT self_element_id INTO item_self_element_id FROM v_item_self_element  WHERE item_id = cable_item_id; 

-- check if any endpoint or its ancestors match the filter using recursive query 

-- limit relationship elements to cable relationship type and specified cable end 

IF ISNULL(cable_end) OR cable_end = ''

THEN

-- cable end not specified

WITH RECURSIVE ancestors as ( 

     SELECT parent_item_id FROM item_element WHERE parent_item_id in ( 

          SELECT parent_item_id FROM item_element WHERE id in ( 

                 SELECT first_item_element_id  

           FROM item_element_relationship ier  

           WHERE ier.relationship_type_id = 4 AND  

        second_item_element_id = item_self_element_id)) 

     UNION  

           SELECT ie.parent_item_id  

           FROM item_element ie, ancestors AS a  

           WHERE ie.contained_item_id1 = a.parent_item_id 

) SELECT count(i.name)  

INTO row_count  

FROM item i, ancestors  

WHERE i.id = ancestors.parent_item_id and i.name like name_filter_value;

ELSE

-- cable end specified

WITH RECURSIVE ancestors as ( 

     SELECT parent_item_id FROM item_element WHERE parent_item_id in ( 

          SELECT parent_item_id FROM item_element WHERE id in ( 

                 SELECT first_item_element_id  

           FROM item_element_relationship ier,  

        item_element_relationship_property ierp,  

property_value pv  

           WHERE ier.relationship_type_id = 4 AND  

        second_item_element_id = item_self_element_id AND 

  ierp.item_element_relationship_id = ier.id AND 

  pv.id = ierp.property_value_id AND 

          pv.value = cable_end)) 

     UNION  

           SELECT ie.parent_item_id  

           FROM item_element ie, ancestors AS a  

           WHERE ie.contained_item_id1 = a.parent_item_id 

) SELECT count(i.name)  

INTO row_count  

FROM item i, ancestors  

WHERE i.id = ancestors.parent_item_id and i.name like name_filter_value;

END IF;

RETURN row_count > 0; 

END//

displaying busy indicator on primefaces wizard component

 Using primefaces 8 to build a java web application, I had trouble figuring out how to display a busy indicator from a primefaces wizard component.  Other views use onstart/oncomplete attributes for button actions to display a "loading dialog" that indicates the application is busy while the action completes.  That approach doesn't work for wizard navigation, because the action is performed asynchronously from the button click.  I found a stack overflow thread with a description of the solution.

In a nutshell, I used the "onnext" attribute of the p:wizard tag to open the busy dialog, and the flowListener method (onFlowProcess()) to close it.  Here are a couple of code snippets.

First the xhtml code for the wizard:

        <p:wizard id="importWizard"
                  flowListener="#{wizardController.onFlowProcess}" 
                  widgetVar="#{rootViewId}"
                  showStepStatus="false" 
                  showNavBar="false"
                  onnext="PF('loadingDialog').show();">

The code for the modal loading dialog:

    <p:dialog modal="true" 
              id="loadingDialog"
              widgetVar="loadingDialog" 
              showHeader="false" 
              styleClass="viewTransparentBackgroundDialog"
              resizable="false">
        <h:outputText value="Loading Results... Please Wait..." />
        <p/>
        <p:graphicImage library="images" name="ajax-loader.gif" />
    </p:dialog>

The java server code:

     public String onFlowProcess(FlowEvent event) {
        String result = onFlowProcessHandler(event);
        SessionUtility.executeRemoteCommand("PF('loadingDialog').hide();");
        return result;
    }

 

sql logging for jsf application in netbeans

To enable SQL logging to the netbeans console for a JSF application running in Payara with EclipseLink, add the following properties to the persistence.xml file: 

<property name="eclipselink.logging.level" value="FINE"/>
<property name="eclipselink.logging.logger" value="DefaultLogger"/>

More information here and here.

On a related note, logging can also be enabled directly in mysql, if that's the underlying database.  Here are some notes about doing that:

log in to mysql as root:

-> use mysql; 

-> SET global log_output = 'table'; 

-> SET GLOBAL general_log = 1;

​

then you can see the logged statements using:

select * from general_log;

or count the number of sql statements:

select count(*) from general_log; 

​

and clean it up with:

truncate general_log; 

​

java method reference

I can never remember the syntax for passing a method reference in Java, so I'm pasting a note about it here.  Here is a decent link explaining the idea.

Here is a method that accepts a method reference as a parameter, and calls that method in its body:

protected ItemDomainEntity findByPath_(String path, Function<ItemDomainEntity, ItemDomainEntity> parentGetterMethod) {

    <snip>
    ItemDomainEntity candidateParent = parentGetterMethod.apply(candidateItem);
    <snip>

}

And then an example of calling that method:

findByPath_(path, ItemDomainMachineDesign::getParentMachineDesign);

dynamic datatable components in primefaces

I'm making notes about this here for future reference. I have an JSF application where I'm using a primefaces datatable component. 

One thing that I don't love about the current approach is that the rows in the table are domain objects in the application, and each column therefore must be a property of the object in order to display the column values. Some of the columns I want to display in the table are not naturally properties of the underlying domain object, but just temporary values associated with the task at hand. So to add them as table columns, I have to define them as properties of the object and I don't always like that. 

 More than once I've wondered if it would be better to just use a list of strings as each row in the table. This way, some of the columns might map to properties of the domain object, while others are temporary task-specific values. Anyway, I'm stashing a couple of notes here about how to do this in case I want to experiment with it later. 

Here is a snippet for displaying lists of strings as the rows for the datatable based on this thread: 

                            <p:dataTable id="#{rootViewId}TableContent"

                                         value="#{wizardController.rows}"

                                         rendered="true"

                                         var="row">    

                                <p:columns value="#{wizardController.rows[0]}" var="column" columnIndexVar="i">

                                    #{row[i]}

                                </p:columns>

And here is a second thread for displaying dynamically selected columns. This still uses properties on the domain object, but I wonder if I could retrieve the values from some sort of map associated with the domain object instead? Here is a snippet of the view components from that thread: 

<h:form>

    <p:selectCheckboxMenu value="#{employeeBean.selectedColumns}"

                          label="Table Columns">

        <f:selectItems value="#{employeeBean.columnMap.entrySet()}"

                       var="entry"

                       itemValue="#{entry.key}"

                       itemLabel="#{entry.value}"/>

        <p:ajax event="change" update="table"/>

    </p:selectCheckboxMenu>

    <br/>

    <p:dataTable id="table" var="emp" value="#{employeeBean.employeeList}">

        <p:columns value="#{employeeBean.selectedColumns}" var="colKey">

            <f:facet name="header">

                <h:outputText value="#{employeeBean.columnMap[colKey]}"/>

            </f:facet>

            <h:outputText value="#{emp[colKey]}"/>

        </p:columns>

    </p:dataTable>

</h:form>

And the corresponding bean code:

@ManagedBean

@ViewScoped

public class EmployeeBean {

    private List<String> selectedColumns = new ArrayList<>();

    private List<Employee> employeeList = new ArrayList<>();

    private Map<String, String> columnMap = new LinkedHashMap<>();

    @PostConstruct

    private void postConstruct() {

        initColumnProperties();

        initEmployeeList();

    }

    private void initColumnProperties() {

        addColumn("id", "ID");

        addColumn("name", "Name");

        addColumn("phoneNumber", "Phone Number");

        addColumn("address", "Address");

        selectedColumns.addAll(columnMap.keySet());

    }

    private void addColumn(String propertyName, String displayName) {

        columnMap.put(propertyName, displayName);

    }

    private void initEmployeeList() {

        DataFactory dataFactory = new DataFactory();

        for (int i = 1; i < 20; i++) {

            Employee employee = new Employee();

            employee.setId(i);

            employee.setName(dataFactory.getName());

            employee.setPhoneNumber(String.format("%s-%s-%s", dataFactory.getNumberText(3),

                    dataFactory.getNumberText(3),

                    dataFactory.getNumberText(4)));

            employee.setAddress(dataFactory.getAddress() + "," + dataFactory.getCity());

            employeeList.add(employee);

        }

    }

    public List<Employee> getEmployeeList() {

        return employeeList;

    }

    public List<String> getSelectedColumns() {

        return selectedColumns;

    }

    public void setSelectedColumns(List<String> selectedColumns) {

        this.selectedColumns = selectedColumns;

    }

    public Map<String, String> getColumnMap() {

        return columnMap;

    }

}